Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· pp. 3527-3537· 0 citations· 14 references
Abstract
Class imbalance is a significant challenge in many practical classification tasks, particularly in federated learning (FL), where both global data imbalance and local data heterogeneity across clients worsen the problem. Addressing the class imbalance problem effectively while adhering to privacy constraints remains a formidable challenge. A common solution is to rebalance (or reweight) the samples of different classes; however, the rebalancing ratio largely depends on empirical results. In this study, we propose an analytical class rebalancing method to compute the optimal rebalancing ratios for imbalanced datasets. We first theoretically derive the relationship between evaluation metrics--such as macro-precision, macro-recall, and macro-F1--and the rebalancing ratio. Based on these findings, we devise an efficient algorithm to determine the optimal rebalancing ratio that maximizes the corresponding metrics. Our method is parameter-free and doesn't increase the complexity of existing neural models. We demonstrate that our method achieves the optimum ratio for maximizing the concerned metrics while maintaining low computational complexity, scaling linearly with both the number of clients and samples. Experimental results on different datasets validate the effectiveness of our algorithms. To the best of our knowledge, we are the first to provide an analytical and parameter-free solution to the problem.
Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that"increasing the number of global models yields better initialization,"and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.
Empirical evaluations on benchmark datasets show that federated SVM framework provides effective results comparable to NN-based FL approaches while significantly less computational and communication overhead.
Deebakkarthi Chinnasame Rani, Gowtham Ramesh, Sountharrajan Sehar et al.· Journal of Computational and...· 0 citations
Deep learning models using traditional softmax classifiers have achieved remarkable success in various classification tasks. However, their performance degrades significantly on imbalanced datasets. Although Balanced Softmax is widely adopted as a state-of-the-art rebalancing method, it possesses inherent limitations, such as yielding disproportionately lower testing accuracy for tail classes. To mitigate these shortcomings, we propose the Class-Balanced Softmax (CBS). Rooted in a theoretical Bayesian framework and a heuristic power-law assumption, the CBS is a simple logit adjustment that is computationally inexpensive and easily integrated into existing pipelines. Furthermore, we characterise a fundamental phenomenon in models trained on imbalanced data, termed the preference issue, wherein models exhibit higher training error and a larger generalisation gap for classes with limited data. To quantify this issue, we introduce a novel metric and demonstrate that CBS effectively mitigates the preference issue. Extensive experiments on large-scale benchmarks show that CBS is highly scalable and outperforms existing methods, including Balanced Softmax.
Yi-Hang Zhu, Rajeev Raman, Shiqi Su et al.· arXiv.org· 0 citations
This paper proposes Federated Learning with Consistency Optimization Algorithms (FedCO), a novel optimization framework that incorporates a label-skew-aware correction loss and neural feature distribution regularization during local training that significantly improves accuracy and convergence under diverse non-IID settings.
Ruiqi Wu, Yehong Li, Hongjie Guo et al.· Computers, Materials & C...· 0 citations
A new FAL framework is proposed that utilizes federated representation learning to align client data in a shared embedding space that achieves performance that surpasses existing FAL methods even when they are given substantially larger annotation budgets, demonstrating the value of centralized coordination under privacy constraints.
Conventional federated learning (FL) relies on parameter averaging, which forces clients to be doubly homogeneous: it demands an identical architecture and degrades under non-IID data. Real-world deployments usually break both assumptions. We sidestep both by building a decentralized knowledge distillation framework in which each client evaluates its peers'model snapshots on its own local data and distills from the resulting soft predictions. Because knowledge is transferred through the shared class posterior, clients are free to run different architectures; and because every teacher is evaluated on the student's own device, raw data never leaves the client, with no central server or public dataset required. Within this setting, we identify and address an under-examined problem: how to combine the peer teacher predictions. Existing methods, like uniform averaging, ignore how knowledge reliability varies across teachers and classes. We propose Class-wise Reliability-Aware Distillation (CRAD), which, per class, first discards teachers that disagree with the peer consensus and then takes a weighted average of the rest, weighting each teacher by its per-class reliability (precision, or inverse variance). Since the variance of an accuracy from $n$ samples scales as $1/n$, support enters automatically: among the teachers that survive filtering, a teacher is trusted for a class to the degree that it is both accurate and well-evidenced for it. On three image-classification benchmarks (CIFAR-10, CIFAR-100, and PathMNIST colon pathology), across heterogeneous architectures under severe non-IID skew, CRAD consistently outperforms competing methods in global accuracy.
Baraa Bilbeisi, Mengchen Fan, Baocheng Geng et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
