Jul 2026· IEEE Transactions on Neural Networks and Learning Systems· Vol PP· 1 citation
Medicine
TL;DR
This article utilizes bias-variance-diversity (BVD) decomposition theory to examine the impact of noisy labels on three mainstream ensemble paradigms: Bagging, Boosting, and Stacking and characterizes the mechanisms behind performance degradation.
Abstract
Ensemble learning methods combine multiple weak base learners to create a robust decision model, effectively analyzing feature-response relationships across various domains. However, the assumption of accurate sample-label relationships in supervised learning often breaks down in real-world datasets, leading to performance degradation due to incorrect information. The framework-specific effects of label noise on ensemble learning have not been sufficiently explored from a theoretical perspective. This article investigates the problem of learning from datasets contaminated by label noise within ensemble frameworks. We utilize bias-variance-diversity (BVD) decomposition theory to examine the impact of noisy labels on three mainstream ensemble paradigms: Bagging, Boosting, and Stacking. Our theoretical analysis characterizes the mechanisms behind performance degradation and guides the development of targeted strategies: data filtering (DF) for Bagging, sample reweighting (SR) for Boosting, and interactive feature purification (IFPS) for stacking. We validate our approaches on synthetic and real-world noisy-label benchmarks, demonstrating consistent improvements over traditional ensemble methods through extensive comparison and ablation experiments. Our findings offer actionable insights for enhancing the robustness of ensemble learning in the presence of noisy labels, thereby broadening its applicability in practical scenarios.
This work conducts a comparative empirical study of five MU methods across symmetric, asymmetric, instance-dependent, and open-set noise on CIFAR-10, CIFAR-100, and the real-world noisy dataset Food-101N and finds that the appropriate unlearning strategy is conditioned on the noise structure.
Multiclass classification is a fundamental problem across a wide range of domains. It is still challenging due to possession of high inter-class similarity, class imbalance datasets, and variability in data distributions. Rule-based classifiers such as XGBoost often achieve stronger performance on structured features, but they are limited in capturing smooth functional relationships among variables. Similarly, neural network models can represent complex nonlinear interactions but frequently suffer from overfitting and generalization issues. To address these limitations, we propose LFS-FRAME, a Leakage-Free Stacked ensemble framework that integrates functional learning using Kolmogorov-Arnold Networks (KAN) and rule-based learning via XGBoost for robust multiclass classification. The proposed framework constructs unbiased meta-features by employing a strict out-of-fold stacking strategy to ensure complete isolation between training and validation data hence preventing performance leakage. By learning over probabilistic outputs from heterogeneous base learners, the meta-classifier effectively exploits both global functional patterns and sharp decision boundaries present in the complex data. Experimental evaluations on multi-class datasets demonstrate that LFS-FRAME improves performance metrics, and overall accuracy is 89.85% in identifying major families and 81.74% in identifying sub-families relative to strong single-model baselines. These results highlight the effectiveness of leakage-free functional and rule-based stacking for reliable and generalizable multiclass classification.
S. P. Sharmila, Aruna Tiwari· arXiv.org· 0 citations
C-Score, a compact framework that evaluates training behavior in three complementary spaces: prediction, feature representation, and optimization, suggests that clean accuracy alone is insufficient for evaluating SSL robustness in open-world environments, and that internal diagnostic signals are necessary for more reliable robustness assessment under unlabeled contamination.
Tsao-Lun Chen, Chicheng Fu, Han-Yi Chou et al.· 0 citations
While SSL outperforms training from scratch on average and remains competitive with state-of-the-art tree ensembles, the SSL-vs-scratch gains exhibit high inter-task variance and lack significance, indicating the findings reflect general properties of tabular SSL rather than idiosyncrasies of one particular pretext task.
Spurious correlations pose a significant challenge to the robustness of modern machine learning. The inherent imbalance in dataset distributions often leads traditional Empirical Risk Minimization (ERM) models to rely on majority spurious attributes for classification, resulting in poor performance on minority groups. This problem becomes particularly challenging when the spurious attributes are unavailable. Existing group-label-free methods often upsample minority groups or misclassified real training examples; repeating the same instances can reduce effective diversity and encourage overfitting. To mitigate these spurious correlations from a data-centric perspective in the absence of prior knowledge, we introduce Subpopulation-Aware Generative Enhancement (SAGE), a two-stage generative augmentation framework. Using cluster-derived sub-labels and class labels, we fine-tune a conditional generative model and text encoder, generating targeted synthetic data to fill underrepresented regions in the training set and construct a balanced validation set for last-layer reweighting. We experimentally show that SAGE achieves 89.5%, 85.7%, and 79.1% worst-group accuracy on Waterbirds, CelebA, and MetaShift, respectively, outperforming the best group-label-free baselines by up to 7.7 percentage points.
The proliferation of high-dimensional data in genomics, text analytics, hyperspectral imaging and industrial sensing has exposed a persistent weakness of conventional classifiers: as the number of features grows far beyond the number of available samples, distance measures lose contrast, decision boundaries become unstable, and models overfit noise rather than signal. This paper proposes an Adaptive Ensemble Learning (AEL) framework that addresses this small-n-large-p regime through three coupled mechanisms. First, relevance-biased stochastic subspace generation constructs diverse yet informative feature views using a composite mRMR-ReliefF ranking, so that base learners are neither confined to the same dominant features nor flooded with noise. Second, a heterogeneous pool of base learners is scored by a competence measure that jointly rewards out-of-bag accuracy, pairwise disagreement and prediction stability, after which redundant or weak members are removed by diversity-aware pruning. Third, ensemble weights are refined iteratively through a temperature-controlled softmax update rather than fixed at training time, allowing the ensemble to reallocate influence as competence estimates sharpen. The framework was evaluated on six benchmark high-dimensional datasets containing between 617 and 12,600 features. AEL attained a mean accuracy of 94.2 per cent, improving on the strongest baseline by 3.5 percentage points, and the gain widened as dimensionality increased. A Friedman test followed by Nemenyi post-hoc analysis confirmed that the improvement is statistically significant at the 0.05 level, while an ablation study showed that all three mechanisms contribute non-trivially to the final result.
Porwal Rabins· International Journal of Inn...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
