This study experimentally measures the impact of non-IID data, noisy data, and fairness in client selection on model accuracy and convergence, and proposes a privacy-preserving scoring method to assess each client's contribution in FL.
Abstract
Federated learning (FL) is a popular distributed learning framework where multiple clients perform local training and a server aggregates the locally updated models. FL enables decentralized training while preserving the privacy of clients'datasets. However, non-independent and identically distributed (non-IID) or noisy datasets can lead to low model accuracy or high convergence latency. Precluding these clients through client selection may mitigate the problem, but heavily biased client selections may also degrade the learning performance. In this study, we first experimentally measure the impact of non-IID data (including skews in data quantity and label distribution), noisy data, and fairness in client selection on model accuracy and convergence. We then propose a privacy-preserving scoring method to assess each client's contribution in FL, with experiments conducted to demonstrate the effectiveness of the proposed assessment.
Federated Learning (FL) allows decentralized clients to train models collaboratively while preserving data privacy. However, distribution mismatch across clients often leads to poor global generalization and degraded local client-level performance. In such scenarios, some of the clients with their local models trained solely on local data may perform better than the globally learnt model, thus nullifying the benefits of collaborative federated learning. To address this, we propose SAPE-FL (Similarity-Aware Personalized Federated Learning), a novel personalization framework that anchors each client's model to both the global model and a similarity-weighted peer averaged model. By incorporating dynamic, client-specific regularization based on both model similarity and output similarity, SAPE-FL adaptively balances global knowledge transfer and peer collaboration while filtering out dissimilar clients. This dual anchoring mitigates negative transfer and enhances robustness in heterogeneous settings. We theoretically analyze our algorithm establishing its convergence guarantees and empirically show that SAPE-FL outperforms state-of-the-art methods under high statistical heterogeneity and low client data regimes.
A. Kumar, Sunil Gupta, Ngyuen Dang et al.· 0 citations
Federated learning (FL) is a promising approach for training distributed machine learning models while preserving clients’ data privacy. However, in real-world FL systems, data are often not independent and identically distributed (non-IID). This heterogeneity can slow convergence, degrade model performance, and increase client drift. To address these challenges, numerous methods have been proposed to mitigate non-IID data effects by optimizing client selection, local training, and model aggregation strategies. Despite their effectiveness in improving performance and efficiency, these methods rarely consider fairness across clients. Improving global accuracy does not guarantee balanced participation, influence, or outcomes, which may lead to biased model behavior across clients. In this survey, we review existing non-IID mitigation methods in FL from a fairness perspective and provide a systematic analysis of their implicit impact on client participation and influence. Unlike prior surveys that treat fairness as a separate research direction, this work analyzes how these methods designed for non-IID mitigation implicitly shape fairness outcomes across clients. Our taxonomy classifies existing methods into three categories—fairness-aware, semi-fairness-aware, and fairness-unaware—based on their design strategies for client selection and model aggregation. Using this taxonomy, we analyze the advantages, trade-offs, and limitations of each category and highlight that mitigating non-IID data does not guarantee fairness across clients. Finally, we identify open challenges and outline future directions, including system-level FL design that jointly considers non-IID mitigation and fairness and the development of standardized fairness evaluation metrics. Overall, this survey aims to provide a structured perspective on the relationship between non-IID mitigation and fairness and support the development of more balanced and scalable FL systems under non-IID conditions.
Mohannad Alsofyani, Isra Al-Turaiki, H. Mathkour· Electronics· 0 citations
Federated low-rank adaptation (FedLoRA) allows multiple clients to collaboratively fine-tune large language models (LLMs) on downstream tasks without exposing their private data. To mitigate privacy leakage during aggregation, differential privacy (DP) is widely used to clip and perturb local model updates with noise, yet it can compromise model accuracy due to the inherent privacy-utility trade-off. The performance degradation becomes worse under the FedLoRA setting with the amplified DP noise impact and client heterogeneity in both model structure and data distribution. In this work, we propose iP-FedLoRA, a privacy-preserving federated fine-tuning framework for heterogeneous clients that strikes a good privacy-utility balance. Specifically, to fully utilize clients' heterogeneous resources, we customize LoRA modules based on their available resources. iP-FedLoRA employs matrix-wise differentially private local fine-tuning with sensitivity-aware noise allocation and rank-compensated LoRA regularization, which effectively alleviates noise impact of low-rank modules and enhances training efficiency. By leveraging noise-resilient knowledge distillation, iP-FedLoRA facilitates heterogeneous LoRA aggregation that selectively prioritizes high-confidence knowledge to filter DP-induced noise, thereby achieving robust knowledge transfer. Through rigorous privacy analysis and extensive experiments, we show that iP-FedLoRA provides privacy guarantees, improves model accuracy by up to 3.8%, and expedites training by 1.37-2.23×.
Nan Yan, Yu-Qing Li, Xiong Wang et al.· Proceedings of the 32nd ACM...· 0 citations
Federated Learning (FL) avoids centralizing raw data, but server-side access to per-client updates still creates a significant privacy risk because gradients can leak sensitive information through inversion and related attacks. A common defense is client-level Differential Privacy (DP), which reduces attack fidelity by perturbing individual updates, but often at the cost of degraded predictive utility, especially in heterogeneous settings. In this paper, we study a different design choice: aggregate-only observability, where the server learns only an aggregate update rather than the individual client contributions that compose it. We develop a unified evaluation framework comparing baseline FL, client-level DP-FL, an idealized sum-only aggregation mechanism (hide_sum), and a quantum-inspired aggregate-observation primitive (qsum) based on finite-shot GHZ-style phase estimation. Experiments on Texas PUDF and MIMIC-IV show that the aggregate-only regimes preserve competitive predictive utility relative to baseline FL while removing the direct per-client observation channel used by standard server-side inversion attacks. We also analyze utility sensitivity to shot budgets and the resource scaling of the quantum-inspired primitive. The results suggest that privacy–utility tradeoffs in FL depend not only on how updates are perturbed, but also on what the server is allowed to observe. Aggregate-only observability therefore provides a useful mechanism-level complement to noise-based privacy defenses, while also motivating hybrid designs to address residual leakage from aggregate updates.
Clifford N. Jones, Md Nahid Hasan, S. Wagle et al.· International Conference on...· 0 citations
This paper considers the practical setting where the learner keeps a small proxy dataset, and proposes a dynamic, influence-aware client selection framework that estimates each client's potential utility to the learner's optimization objective using proxy influence signals on a learner-specific proxy set.
A novel FL framework is presented, FedPhoenix, that stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features.
Jia-Hao Wu, Ming Hu, Yanxin Yang et al.· Neural Information Processin...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
