Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· pp. 4916-4926· 0 citations· 8 references
Abstract
Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, but its performance degrades significantly under heterogeneous data distributions. While Clustered Federated Learning (CFL) mitigates heterogeneity by grouping similar clients, most existing methods primarily address label distribution skew and overlook feature distribution skew, where clients share labels but differ systematically in input characteristics. In this paper, we reveal a key observation: under feature distribution skew, different network layers exhibit highly unequal sensitivity, with early layers encoding strongly client-specific feature patterns that dominate aggregation behavior. Motivated by this, we propose SensCluster, a novel sensitivity-aware CFL framework that constructs compact client representations by selecting parameters that are most responsive to local feature distributions. By leveraging Fisher information, SensCluster identifies and retains only the most influential parameters, constructing sensitivity-aware representations for clustering while reducing interference from irrelevant features. Extensive experiments on both image (FEMNIST, CIFAR10-c, CINIC10-c, EuroSAT) and time-series (HAR) benchmarks demonstrate that SensCluster consistently outperforms state-of-the-art CFL methods across diverse feature skew scenarios.
Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space using an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client.
Mirko Konstantin, S. Zachow, Anirban Mukhopadhyay· 0 citations
Experimental results show that routing-aware collaboration consistently improves personalized performance compared to conventional federated averaging and local training, while maintaining the same communication cost, and shows that client-centric and expert-centric clustering provides an effective and scalable approach for personalized federated instruction fine-tuning of sparse MoE LLMs.
Ankita Sharma, B. Farahani, S. Moosavi et al.· 0 citations
Federated Learning (FL) allows decentralized clients to train models collaboratively while preserving data privacy. However, distribution mismatch across clients often leads to poor global generalization and degraded local client-level performance. In such scenarios, some of the clients with their local models trained solely on local data may perform better than the globally learnt model, thus nullifying the benefits of collaborative federated learning. To address this, we propose SAPE-FL (Similarity-Aware Personalized Federated Learning), a novel personalization framework that anchors each client's model to both the global model and a similarity-weighted peer averaged model. By incorporating dynamic, client-specific regularization based on both model similarity and output similarity, SAPE-FL adaptively balances global knowledge transfer and peer collaboration while filtering out dissimilar clients. This dual anchoring mitigates negative transfer and enhances robustness in heterogeneous settings. We theoretically analyze our algorithm establishing its convergence guarantees and empirically show that SAPE-FL outperforms state-of-the-art methods under high statistical heterogeneity and low client data regimes.
A. Kumar, Sunil Gupta, Ngyuen Dang et al.· 0 citations
This paper proposes Federated Learning with Consistency Optimization Algorithms (FedCO), a novel optimization framework that incorporates a label-skew-aware correction loss and neural feature distribution regularization during local training that significantly improves accuracy and convergence under diverse non-IID settings.
Ruiqi Wu, Yehong Li, Hongjie Guo et al.· Computers, Materials & C...· 0 citations
Experiments under representative Non-IID settings on benchmark datasets show that PFLS-One achieves improved accuracy and faster convergence compared with representative baseline methods, and the convergence analysis under a non-convex objective provides theoretical support for the proposed method.
Federated learning enables privacy-preserving collaboration across distributed devices without centralizing local data. However, clients may differ not only in data distributions but also in domain knowledge and annotation capabilities. In this paper, we introduce label granularity skew, a new form of statistical heterogeneity in federated hierarchical classification, in which clients provide taxonomy-consistent labels at different levels of detail within a shared class hierarchy. To model this heterogeneity, we generate client-specific local label hierarchies using a probabilistic relational neighbor classifier and construct a WordNet-guided hierarchy via silhouette score-based coarsening. Our analysis shows that strongly coupled hierarchical models are sensitive to incomplete supervision, while the conditional softmax classifier is more robust. Based on this insight, we propose Branch-wise Decoupled Fine-Tuning (BDFT) and its federated version, FedBDFT, which fine-tune branch-wise classifiers and aggregate them through federated optimization. Experiments on CIFAR-100, TinyImageNet, and ImageNet show that FedBDFT substantially improves robustness under severe label granularity skew, with average gains of 27.9% and 56.4% at skewness levels of 0.6 and 0.9, respectively. Zero-shot results further indicate that FedBDFT better preserves hierarchical representations for unseen fine-grained classes. These findings demonstrate its effectiveness for federated hierarchical classification with heterogeneous label granularities.
Jaeheon Kim, Hokeun Kim, B. Choi· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
