Differential Privacy for Federated Learning with Adaptive Noise

Abstract

Federated learning (FL) presents a promising paradigm for training machine learning models on decentralized data sources without directly exchanging the data itself. However, the inherent privacy risks associated with aggregating model updates introduce significant challenges. Traditional differential privacy (DP) techniques often rely on adding uniform random noise to gradients, which can be overly conservative and degrade model accuracy. This paper introduces an adaptive noise scheme for FL that dynamically adjusts the noise level based on the sensitivity of the aggregated gradients. The proposed method monitors gradient variance and employs a stochastic gradient descent (SGD) variant with a dynamically adjusted learning rate and noise scale. We demonstrate through theoretical analysis and a simplified simulation that this approach significantly reduces the overall noise level compared to standard DP while maintaining a comparable privacy guarantee, ultimately leading to improved model accuracy in FL settings. The key contribution lies in the intelligent adaptation of noise, responding directly to the data's inherent characteristics.