Information-Theoretic Foundation for Trustworthy Federated Learning

Abstract

Federated Learning (FL) offers a promising approach to training machine learning models on decentralized data while preserving data privacy. However, the inherent heterogeneity of data across clients and potential malicious behavior introduce challenges to the trustworthiness of FL systems. This work proposes a novel information-theoretic framework for quantifying trust in FL, moving away from traditional subjective trust assumptions. We define a "trust score" based on the mutual information between local model updates and a global consensus model. This score directly measures the contribution of each client's update to the collective knowledge, providing a rigorous metric for assessing model divergence and identifying potential outliers. The core contribution of this paper lies in providing a quantifiable and objective method for evaluating trust in FL, enabling more robust and reliable deployments. We explore the theoretical properties of this mutual information-based trust score and demonstrate its potential for improved outlier detection and model convergence in federated learning scenarios. The framework utilizes concepts from information theory, specifically mutual information, to provide a mathematically grounded approach to assessing the quality and trustworthiness of decentralized model training.