GBLSTSVM: Granular Ball Least Squares Twin SVM

A from-scratch Python implementation of the Granular Ball Least Squares Twin SVM (GBLSTSVM), based on the closed-form optimization formulation proposed by Tanveer et al. (2025). Includes a kernelized variant (KGBLSTSVM) using Nystroem approximation, and a rigorous 5-fold stratified cross-validation benchmark against a standard linear SVM.

Standard SVMs treat every training point individually. GBLSTSVM instead first compresses the data into granular balls — clusters of nearby, same-class points represented by a center and radius — and fits two non-parallel hyperplanes (one per class) directly against these balls. This reduces sensitivity to noise and outliers and can significantly cut the number of "points" the optimizer has to reason about.

The two hyperplanes are found via a closed-form least-squares solve (no iterative QP solver needed), which is what makes the least-squares variant of twin SVM fast. KGBLSTSVM extends this to non-linear boundaries using an RBF kernel, approximated via the Nystroem method for tractability.

Reference: Tanveer, M., Sharma, R. K., Quadir, A., & Sajid, M. (2025). Enhancing robustness and efficiency of least square twin SVM via granular computing. Pattern Recognition, 112021. https://doi.org/10.1016/j.patcog.2025.112021

git clone https://github.com/Nexxumm/Linear-GBLSTSVM.git
cd Linear-GBLSTSVM
pip install -r requirements.txt
python evaluate.py

5-fold stratified cross-validation on the UCI Breast Cancer (Ljubljana) dataset (286 samples, 9 features, 201/85 class split):

Takeaway: all three models land at roughly the same accuracy, but GBLSTSVM achieves the best F1 with the lowest fold-to-fold variance — meaning it's the most consistently balanced classifier at catching the minority (recurrence) class, not just the luckiest on one split. KGBLSTSVM performs comparably on average but with higher variance, likely due to the small number of granular balls (67) limiting how much the Nystroem approximation can be trusted.

This is the UCI Breast Cancer (Ljubljana) dataset — not the more commonly cited Wisconsin Diagnostic dataset, where 95%+ accuracy is typical. Ljubljana is a smaller, noisier dataset (mostly categorical clinical features), and published results across many classifiers, including well-tuned SVMs, generally sit in the 70-77% range. The numbers above should be read against that ceiling, not against Wisconsin-style benchmarks.

A few real issues came up while getting these results, worth documenting since they're not obvious from the code alone:

• The paper's reported hyperparameters (c1=1000, c2=0.00001) caused GBLSTSVM to collapse to predicting a single class on this pipeline. Their values were tuned for a differently-configured setup (unscaled features, different granular ball purity threshold, single fixed split vs. 5-fold CV). Re-tuning via grid search — while explicitly checking prediction distributions, not just accuracy — found (c1=1, c2=10) avoided collapse and gave the best accuracy/F1 trade-off.
• KGBLSTSVM's default n_components=100 exceeded the actual number of granular balls available (67), causing Nystroem to use every ball center as a kernel landmark instead of approximating. This made the resulting Gram matrix severely ill-conditioned (condition number ≈ 4.4 billion), to the point where c1/c2 had almost no effect on the solution regardless of their value. Fixed by reducing n_components to 20 and increasing the solver's regularization term (eps) from 1e-6 to 1e-3.

c1/c2 (and gamma, n_components for the kernel variant) were selected via grid search, optimizing for F1 while explicitly rejecting any combination where the model collapsed to predicting a single class — a failure mode that can otherwise look deceptively good on accuracy alone given the dataset's class imbalance.

MIT