Jul 2026· International Conference on Big Data Computing Service and Applications· pp. 9-23· 0 citations· 68 references
Abstract
Federated Learning (FL) trains shared models across distributed clients without pooling raw data, but its behaviour under heterogeneous data distributions and the associated privacy risks remain imperfectly understood. This paper presents a controlled empirical comparison of Federated Averaging (FedAvg), Federated Proximal (FedProx), Federated Adaptive Moment Estimation (FedAdam), and Differentially Private Federated Averaging (DP-FedAvg) under matched conditions. The experiments use CIFAR-10 and MNIST with Non-Independent and Identically Distributed (non-IID) client partitions, including Dirichlet and class-restricted partitioning. Each condition is repeated across 20 independent seeds. We examine how optimisation choice shapes convergence and stability, how differential privacy affects classification performance, and how privacy exposure changes over the course of training. We find that FedAdam improves performance under severe client heterogeneity, where client gradients conflict strongly. FedProx changes the training trajectory, but we do not find it to improve cross-seed stability. DP-FedAvg reduces the success of gradient-reconstruction and membership-inference attacks, but this protection comes with a measurable reduction in classification performance due to gradient clipping and additive noise. A key finding is that privacy exposure is temporally concentrated rather than uniform across training. It is highest in the earliest rounds, when gradients are largest and most informative, and declines as training progresses. These results suggest that private FL systems should account for training dynamics, with particular emphasis on protection during early training rounds.
Federated learning is a decentralised machine-learning approach in which several clients jointly build a shared model without moving their raw data to one location. Rising concerns around privacy, tightening regulation, and restrictions on how data may be owned or shared have made this approach increasingly attractive in practice. Although federated learning lowers privacy exposure relative to centralised training, deploying it in practice is complicated by clients whose data are unevenly distributed and non-identically distributed, by clients that participate inconsistently, and by training that can converge unpredictably. To obtain global models that train reliably and consistently even when client data are heterogeneous, this work puts forward a federated learning system built around privacy preservation. The design follows a client–server pattern in which a coordinating server aggregates updates from local models using weights that account for imbalance among participants. The behaviour of the resulting system is examined methodically across several data-distribution regimes — IID, mildly non-IID, and severely non-IID. The experiments show that the framework converges reliably and delivers predictive accuracy that holds up well, especially in the more difficult non-IID cases. Compared with conventional federated learning baselines, the approach shows greater robustness and steadier performance across successive training rounds. Because it is simple to implement, repeatable, and built with real deployment in mind, the architecture suits privacy-sensitive, decentralised use cases such as distributed intelligent systems, industrial monitoring, and healthcare analytics.
Shyam Patel, S. Khan· 2026 International Conferenc...· 0 citations
This study implemented a comprehensive experimental framework for analysing FL performance using standard FL aggregation protocols FedAvg, FedProx, and SCAFFOLD in conjunction with Differential Privacy mechanisms; specifically, the Gaussian noise mechanism with Rényi Differential Privacy (RDP) accountants.
Gradient-boosted trees outperform neural networks on tabular data without privacy and off-path privatization and sequential noise accumulation explain the behavior; boosting’s main advantage is not accuracy but communication, achieving one to three orders of magnitude fewer values per client.
Federated low-rank adaptation (FedLoRA) allows multiple clients to collaboratively fine-tune large language models (LLMs) on downstream tasks without exposing their private data. To mitigate privacy leakage during aggregation, differential privacy (DP) is widely used to clip and perturb local model updates with noise, yet it can compromise model accuracy due to the inherent privacy-utility trade-off. The performance degradation becomes worse under the FedLoRA setting with the amplified DP noise impact and client heterogeneity in both model structure and data distribution. In this work, we propose iP-FedLoRA, a privacy-preserving federated fine-tuning framework for heterogeneous clients that strikes a good privacy-utility balance. Specifically, to fully utilize clients' heterogeneous resources, we customize LoRA modules based on their available resources. iP-FedLoRA employs matrix-wise differentially private local fine-tuning with sensitivity-aware noise allocation and rank-compensated LoRA regularization, which effectively alleviates noise impact of low-rank modules and enhances training efficiency. By leveraging noise-resilient knowledge distillation, iP-FedLoRA facilitates heterogeneous LoRA aggregation that selectively prioritizes high-confidence knowledge to filter DP-induced noise, thereby achieving robust knowledge transfer. Through rigorous privacy analysis and extensive experiments, we show that iP-FedLoRA provides privacy guarantees, improves model accuracy by up to 3.8%, and expedites training by 1.37-2.23×.
Nan Yan, Yu-Qing Li, Xiong Wang et al.· Proceedings of the 32nd ACM...· 0 citations
Federated Learning (FL) enables collaborative model training without sharing raw data, but its effectiveness degrades under Non-IID client data and inefficient participation. In such settings, FedAvg may exhibit slow convergence, high variance across rounds, and elevated communication overhead. We propose a clustered ensemble framework to improve training stability and communication efficiency under label-skewed Non-IID distributions. Clients are clustered by data similarity, and one model is trained per cluster. At inference time, we employ a confidence-based ensemble that selects (or combines) cluster models, while adaptive client selection regulates participation within clusters to reduce redundant communication. Experiments on MNIST, Fashion-MNIST, CIFAR-10, and SVHN show more stable learning dynamics and lower communication cost while maintaining competitive accuracy compared to FedAvg. We also analyze how participation policies affect convergence behavior and client fairness.
Artur Sousa Freitas, A. T. Akabane, J. Estrella· International Conference on...· 0 citations
This study experimentally measures the impact of non-IID data, noisy data, and fairness in client selection on model accuracy and convergence, and proposes a privacy-preserving scoring method to assess each client's contribution in FL.
