Sparsity in Federated Learning: A Survey
TL;DR
A novel taxonomy of sparse FL methods is introduced that systematically organizes the existing literature according to their core objectives and methodological choices, and identifies open challenges, expose research gaps, and extract guidance to help practitioners understand and adopt sparsity mechanisms in FL.
Abstract
Conventional Federated Learning (FL) pipelines focus on the collaborative training of a global dense model across client devices. Sparsity has been increasingly adopted in FL, during or after local optimization, for a range of objectives, including reducing communication and computation costs, supporting unlearning, enhancing privacy guarantees, and improving local personalization. In this survey, we introduce a novel taxonomy of sparse FL methods that systematically organizes the existing literature according to their core objectives and methodological choices. Using this taxonomy, we analyze and categorize prior work, highlighting the underlying intuitions, technical mechanisms, benefits, and limitations of each class of approaches. Finally, we identify open challenges, expose research gaps, and extract guidance to help practitioners understand and adopt sparsity mechanisms in FL.