Skip to content
Open access

When Are Linear Classifiers Sufficient? A Re-evaluation and Conditional Boundary Exploration Based on OpenML

Nov 2026 · Frontiers in Computing and Intelligent Systems · 0 citations · 18 references

Abstract

The point at which a linear classifier stops being "sufficient" lacks a quantitative definition. To address this, we evaluated 1,687 records from 298 OpenML datasets across three model families. Using a practical significance framework, we defined linear sufficiency as an accuracy drop within a tolerance threshold δ. At δ=0.02, linear sufficiency was observed in 68.9% of SVM datasets, 62.4% of NN datasets, but only 46.1% of DT datasets. Shallow decision tree extraction highlighted low feature-to-instance ratios and small class counts as primary predictors. However, leave-one-out validation exposed a clear split. The extracted rules worked well for the DT family (0.686 accuracy vs. 0.539 baseline) but failed to surpass baselines for SVM and NN. This suggests that while simple meta-features can predict linear sufficiency for decision trees, they cannot fully capture the complex boundaries of SVMs and neural networks.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.