Learning Where the Data Lives: A Narrative Review of Federated Learning from Differential Privacy to the Open Problems

Abstract

Federated learning---training a shared model across many devices that never surrender their data---inverted machine learning's architecture: instead of data to the model, the model to the data. This article presents a narrative review of that arc's canonical line: Dwork and colleagues' 2006 differential privacy, Dwork and Roth's 2014 foundations, Shokri and Shmatikov's 2015 privacy-preserving deep learning, Konecny and colleagues' 2016 communication strategies, McMahan and colleagues' 2017 FedAvg, Bonawitz and colleagues' 2017 secure aggregation, Zhao and colleagues' 2018 non-IID study, Bonawitz and colleagues' 2019 scale design, Yang and colleagues' 2019 concept paper, Zhu, Liu, and Han's 2019 gradient leakage, Li and colleagues' 2020 convergence analysis, and Kairouz and colleagues' 2021 open problems. The synthesis is organized around three themes: privacy, in which differential privacy's calculus and secure aggregation made learning without exposure precise; algorithm, in which FedAvg's weighted averaging met the heterogeneity of real devices and real data; and system, in which scale deployments faced stragglers, leakage, and the statistical reality of non-IID partitions. It is concluded that federated learning is privacy engineering's rare full-stack success---its limits as precisely mapped as its promise---and that its open problems are the field's charter: heterogeneity, security, and the economics of participation.