SeqFedRPC: Sequential Federated Learning With Regularized Parameter Clustering
TL;DR
This work proposes SeqFedRPC, a novel model decoupling based SFL framework with regularized parameter clustering, and employs a clustering‐based scheme to adaptively decouple the model parameters into shared and personalized subsets, thereby addressing the challenge of non‐IID data.
Abstract
Sequential federated learning (SFL) enables collaborative model training across clients in a chain manner, providing communication‐efficient benefits over traditional all‐gather parallel federated learning (PFL). However, SFL training often suffers from slow convergence and performance degradation due to nonidentically distributed (non‐IID) data distribution. In motivation experiment, we find that model decoupling by partitioning the model into shared and personalized parameters and using just a few personalized parameters with large gradients can improve SFL training performance. Based on above findings, we propose SeqFedRPC, a novel model decoupling based SFL framework with regularized parameter clustering. We introduce a regularization term to promote parameter sparsification and amplify gradient differences, which aids in gradient‐based parameter clustering. Then, we employ a clustering‐based scheme to adaptively decouple the model parameters into shared and personalized subsets, thereby addressing the challenge of non‐IID data by adapting global knowledge with shared parameters and client‐specific distributions with personalized parameters. Extensive experiments on eight benchmark datasets demonstrate that SeqFedRPC surpasses eight SOTA methods, with each client personalizing less than 10% of the total parameters on all datasets at α=0.1$$ \alpha =0.1 $$ .