Distributed Differential Privacy with Federated Learning via Lagrangian Relaxation

Abstract

Achieving strong differential privacy guarantees within the constraints of federated learning, especially when utilizing complex models, remains a significant challenge. This work proposes a novel framework leveraging Lagrangian relaxation to address this issue. The core idea involves incorporating a Lagrangian term directly into the federated learning objective function to formally represent the differential privacy constraint. This allows for an iterative solution of the resulting Lagrangian problem via distributed optimization, providing a controllable mechanism for balancing privacy and model accuracy. The proposed approach offers a more practical and scalable solution compared to existing methods, particularly in scenarios where precise control over the privacy-accuracy trade-off is desired. The effectiveness of this method is demonstrated through theoretical analysis and conceptual discussion, outlining a pathway for future research and implementation.