Information-Theoretic Framework for Trustworthy Federated Learning

Abstract

Federated Learning (FL) offers a promising paradigm for decentralized machine learning, enabling collaborative model training without direct data sharing. However, traditional privacy-preserving techniques within FL often rely on ad-hoc assumptions and lack a rigorous theoretical basis. This work introduces an information-theoretic framework to address this limitation. We define a "Privacy Loss Function" predicated on mutual information between local models and global updates, providing a quantifiable measure of information leakage. The framework leverages established techniques such as differential privacy and homomorphic encryption to minimize this loss, ultimately leading to more robust and trustworthy FL systems. Our approach moves beyond intuitive notions of privacy, offering a mathematically sound foundation for designing and analyzing FL protocols, facilitating the development of truly secure and efficient distributed learning solutions. The core contribution is the formalization of privacy risk in FL using information-theoretic principles, enabling a more precise understanding and control over data leakage.