Decentralized Federated Learning with Differential Privacy via Lagrangian Relaxation

Abstract

Federated learning (FL) offers a promising paradigm for training machine learning models on decentralized data sources without directly exchanging data. However, achieving robust privacy guarantees alongside high model accuracy remains a significant challenge. This paper introduces a novel approach to decentralized federated learning leveraging Lagrangian relaxation and differential privacy. Our method allows for a more nuanced and adaptive trade-off between privacy and accuracy by optimizing a Lagrangian function at each participating node. This decentralized optimization process enables a flexible solution to the inherent tension between privacy preservation and model performance. We demonstrate the effectiveness of this approach through a theoretical analysis and outline a potential implementation strategy. The key innovation lies in the localized optimization of the Lagrangian, mitigating the communication bottlenecks often encountered in centralized FL and offering a pathway towards truly decentralized privacy-preserving learning.