The Decentralized Classroom: A Narrative Review of Federated Learning from Google's Keyboards to the Privacy's Frontier

Abstract

Federated learning---the artificial intelligence whose subject is the decentralized classroom and whose lesson is the model's travel---moved from Dwork's 2006 differential privacy and Shokri and Shmatikov's 2015 gradients through Konečný's 2016 compression, McMahan's 2017 FedAvg, and Bonawitz's 2017 aggregation to Zhao's 2018 non-IID, Kairouz's 2021 survey, and Zhu's 2019 leakage. This article presents a narrative review of that arc's canonical line: Dwork's 2006 ICALP, Shokri and Shmatikov's 2015 CCS, Konečný and colleagues's 2016 strategies, McMahan, Moore, Ramage, Hampson, and Arcas's 2017 FedAvg, Bonawitz and colleagues's 2017 secure aggregation, Zhao and colleagues's 2018 non-IID, Hard and colleagues's 2018 keyboard, Zhu, Liu, and Han's 2019 gradients, Yang and colleagues's 2019 concept, Li and colleagues's 2020 convergence, Li and colleagues's 2020 challenges, and Kairouz and colleagues's 2021 advances. The review is organized around three themes: the privacy's premise and the communication's bottleneck, in which the Dwork's noise and the Shokri-Shmatikov's gradients founded the distributed's training; the algorithm's and the deployment's era, in which the FedAvg's averaging, the secure's aggregation, and the keyboard's deployment gave the federation its engine; and the heterogeneity's and the frontier's era, in which the non-IID's data, the gradient's leakage, the convergence's proofs, and the open's problems carried the field into the privacy's science. It is concluded that federated learning is the machine learning's decentralization---and that its arc is the classroom's reading from the centralized's server to the privacy's frontier.