Differential Privacy Enabled Cascaded Filter for Efficient and Privacy-Preserving Federated Learning

Abstract

Federated learning (FL) enables collaborative model training across multiple clients in a privacy-preserving manner. However, the employment of homomorphic encryption algorithms might lead to high computational cost while the application of differential privacy (DP) methods would sacrifice model performance. To establish efficient and secure FL system as well as maintaining competitive performance, we introduce a DP-enabled cascaded filter with novel model-aggregation mechanism. Taking the model parameters of one client for example, dimensions with large absolute values and significant variations are selected by the cascaded filter and regarded as important dimensions. After that, random noise are added to these important dimensions for data security. Finally, the model parameters will be sent to the central server for aggregation. Unlike traditional DP-based approaches, our method considers each dimension’s informational importance, i.e., both absolute value and its variation. We theoretically prove the convergence of our method and verify the effectiveness on different scenarios including four datasets. The experimental results suggest that our method outperforms the other ones from literature under both IID and non-IID conditions while achieving high level of efficiency and privacy protection performance.