When Does Cost-Sensitive Weighting Matter? A Classifier-Capacity Anal-ysis for Imbalanced Classification

Abstract

Class weighting is the most widely used cost-sensitive remedy for class imbalance, and inverse-frequency “balanced” weights are often applied as an unquestioned default. This paper asks whether the choice among competing weighting mechanisms actually matters, and for which classifiers. Seven mechanisms, namely Balanced, Cost-Matrix, Sqrt-InverseFreq, Log-Damped, Effective-Number, a Search-Tuned power scheme, and focal loss, were evaluated on twenty-four benchmark datasets whose imbalance ratios reach 85:1, under three base classifiers of increasing representational capacity: logistic regression, a soft-voting ensemble of logistic regression, random forest, and a support vector machine, and XGBoost. Using leakage-free nested cross-validation and the Friedman–Nemenyi procedure applied separately to each classifier, mechanism choice produced large, highly significant differences under logistic regression (p < 0.0001), differences that shrank to non-significance under the voting ensemble (p = 0.179) and became indistinguishable from a random ordering under XGBoost (p = 0.955, average-rank spread of 0.46). At the same time, the two high-capacity classifiers outperformed logistic regression under every mechanism tested. The evidence supports a simple two-tier policy: tune the weighting scheme carefully for linear learners, and keep the plain balanced default for ensemble and boosting learners, where additional tuning yields no benefit that survives a significance test.