It is demonstrated that round-wise, layer-wise adaptation can improve the privacy-accuracy-efficiency trade-off of differentially private federated learning.
Abstract
Differentially private federated learning must balance privacy protection against model accuracy and training efficiency. Static gradient clipping applies a fixed threshold throughout training and across model layers, which can cause excessive clipping when the threshold is too small or unnecessarily large noise when it is too large. This paper presents DDP-SA-adaptive, an adaptive gradient clipping and noise adding mechanism for differentially private federated learning with secure aggregation. At each communication round, every client determines a separate clipping threshold for each model layer from the median of its per-sample gradient norms. The resulting layer-wise thresholds adapt to the evolving gradient distributions and calibrate the Laplace noise added before the updates are encoded and secret-shared among intermediate aggregation servers. We evaluate the proposed mechanism on a federated regression task in terms of efficiency, accuracy, privacy, convergence, clipping norm, and noise magnitude. Compared with the static DDP-SA baseline, DDP-SA-adaptive reduces the number of communication rounds by 6.81%, total training time by 19.21%, and average per-round training time by 13.33%, leading to improved training efficiency. It also reduces test loss by 98.74% and increases test R2 by 3.41%, leading to improved model accuracy. To attain R2 = 0.99, the adaptive mechanism operates with a privacy budget of approximately epsilon = 0.1, compared with epsilon = 0.4 for static DDP-SA, thus providing stronger privacy protection and achieving stronger privacy guarantees. These results demonstrate that round-wise, layer-wise adaptation can improve the privacy-accuracy-efficiency trade-off of differentially private federated learning.
FedDecouple is proposed, a phase-decoupled differentially private federated learning framework that is analytically suited for resource-constrained mobile devices and significantly outperforming client-side noised DP-SGD on MNIST and CIFAR-10.
Federated Learning (FL) enables distributed training while keeping data local, but exchanged model updates can leak information through membership inference attacks. Differential privacy mitigates this risk via noise injection; however, aggressive DP regimes with strong noise can destabilize large models. An SNR-guided framework is introduced to select model dimensionality based on the signal-to-noise ratio imposed by the privacy budget. Three optimizers, DP-FedAvg, DP-FedAvgM, and DP-FedAdam, are evaluated across six domains, including image, clinical, IoT, and network security tasks. Privacy leakage is assessed using both loss-based membership inference and the likelihood-ratio attack LiRA. DP-FedAvgM achieves 98.10% accuracy on MNIST at ε =200 with LiRA AUC near random guessing (0.491). SNR-guided models reduce communication cost by up to 66×. Sensitivity calibration experiments further show that incorrect noise allocation can reduce accuracy by up to 1.36 percentage points. These results highlight the importance of model sizing and noise calibration for reliable privacy-preserving FL under strong DP constraints.
Mohammed Hamza, I. B. Sofi, Kuljeet Kaur et al.· International Conference on...· 0 citations
Federated learning (FL) enables collaborative model training across multiple clients in a privacy-preserving manner. However, the employment of homomorphic encryption algorithms might lead to high computational cost while the application of differential privacy (DP) methods would sacrifice model performance. To establish efficient and secure FL system as well as maintaining competitive performance, we introduce a DP-enabled cascaded filter with novel model-aggregation mechanism. Taking the model parameters of one client for example, dimensions with large absolute values and significant variations are selected by the cascaded filter and regarded as important dimensions. After that, random noise are added to these important dimensions for data security. Finally, the model parameters will be sent to the central server for aggregation. Unlike traditional DP-based approaches, our method considers each dimension’s informational importance, i.e., both absolute value and its variation. We theoretically prove the convergence of our method and verify the effectiveness on different scenarios including four datasets. The experimental results suggest that our method outperforms the other ones from literature under both IID and non-IID conditions while achieving high level of efficiency and privacy protection performance.
Zhiqiang Chen, Yuchen Jiang, Ray Y. Zhong et al.· IEEE Transactions on Informa...· 0 citations
Federated low-rank adaptation (FedLoRA) allows multiple clients to collaboratively fine-tune large language models (LLMs) on downstream tasks without exposing their private data. To mitigate privacy leakage during aggregation, differential privacy (DP) is widely used to clip and perturb local model updates with noise, yet it can compromise model accuracy due to the inherent privacy-utility trade-off. The performance degradation becomes worse under the FedLoRA setting with the amplified DP noise impact and client heterogeneity in both model structure and data distribution. In this work, we propose iP-FedLoRA, a privacy-preserving federated fine-tuning framework for heterogeneous clients that strikes a good privacy-utility balance. Specifically, to fully utilize clients' heterogeneous resources, we customize LoRA modules based on their available resources. iP-FedLoRA employs matrix-wise differentially private local fine-tuning with sensitivity-aware noise allocation and rank-compensated LoRA regularization, which effectively alleviates noise impact of low-rank modules and enhances training efficiency. By leveraging noise-resilient knowledge distillation, iP-FedLoRA facilitates heterogeneous LoRA aggregation that selectively prioritizes high-confidence knowledge to filter DP-induced noise, thereby achieving robust knowledge transfer. Through rigorous privacy analysis and extensive experiments, we show that iP-FedLoRA provides privacy guarantees, improves model accuracy by up to 3.8%, and expedites training by 1.37-2.23×.
Nan Yan, Yu-Qing Li, Xiong Wang et al.· Proceedings of the 32nd ACM...· 0 citations
Differential privacy (DP) mechanisms have been widely adopted in federated learning (FL) to enhance model security. However, existing approaches predominantly employ uniform privacy budgets, neglecting personalized requirements arising from heterogeneous user privacy preferences. Such uniform privacy configurations typically necessitate compliance with the most stringent budget, which not only leads to the wasteful underutilization of privacy budgets for certain clients but also compromises overall model utility. To address this limitation, we propose FedSPA, a Subspace Projection Aggregation personalized differential private Federated learning framework. The proposed method conducts singular value decomposition operations on noise-perturbed local models to extract singular value vectors as compact representations of both model structure and privacy noise. The server then clusters clients and identifies a consensus subspace for projecting models with varying noise levels, ultimately aggregating the global model through a residual-aware mechanism. This method not only effectively guides the aggregation of client personalized differential privacy but also reduces communication overhead. Extensive experiments demonstrate the model's effectiveness. Additionally, we provide theoretical proof of the privacy and convergence of FedSPA. Experimental results also showcase its superior performance over personalized DP-FL baselines.
Tianchi Liao, Xiaojun Deng, Lele Fu et al.· Proceedings of the 32nd ACM...· 0 citations
SURGE targets empirical attack resistance under the honest-but-curious server model, rather than a formal privacy guarantee, and drives membership inference performance close to random guessing and substantially degrades the quality of gradient inversion reconstructions.
