Adaptive Differential Privacy for Federated Learning via Randomized Compression

Abstract

Federated learning (FL) offers a promising paradigm for training machine learning models on decentralized data sources without directly exchanging data. However, achieving strong differential privacy (DP) guarantees in FL is a significant challenge, particularly when combined with compression techniques commonly employed to reduce communication costs. Traditional approaches often rely on fixed compression ratios, potentially sacrificing accuracy for privacy or vice versa. This paper introduces an adaptive differential privacy framework for FL that dynamically adjusts the compression ratio based on the sensitivity of the data and the desired privacy level. We leverage randomized compression methods, specifically noise addition with learned parameters, to achieve this adaptation. Our approach demonstrates a novel balance between privacy and utility, offering a more effective solution compared to static compression strategies. The core claim of this work is that achieving strong differential privacy guarantees in federated learning necessitates careful management of compression ratios. The key mechanism involves introducing an adaptive compression scheme. This paper presents a formalization of this adaptive approach, outlining its mathematical foundations and providing a framework for its implementation.