Aug 2026· IEEE/CAA Journal of Automatica Sinica· Vol 13, pp. 1938-1951· 0 citations· 62 references
Abstract
Federated learning aims to provide security for client data privacy in practical machine learning applications. In principle, a global server aggregates the models produced by local clients to obtain a global model. However, the server is challenged when collaborating with local clients handling non-identically distributed data without authorisation to access it. Therefore, advanced solutions advocate the use of generative modules to deliver surrogate data to local clients during a server-agent interaction, without revealing private particulars. We argue that such a unidirectional transfer of surrogate patterns cannot fully represent and harmonise knowledge during the server-client interactions. To this end, we propose a knowledge-imparting generative modelling framework (FedKIG) based on adversarial feature learning and bidirectional knowledge distillation, to explore the potential of interactive generative modelling. In particular, Fed-KIG trains a feature discriminator for each local client to identify the surrogate patterns extracted by the global model. Under the supervision of the local feature discriminators, the server learns a global generator to generate pseudo samples that convey its global perspective. In this manner, local models are enabled to absorb global knowledge, thereby mitigating the training data divergence caused by data heterogeneity. In addition, we develop a bidirectional knowledge distillation strategy to support the entire learning process. This strategy breaks the rigidity of federated distillation by updating knowledge transfer between the server and the clients iteratively, thus overcoming the learning-forgetting issue. The proposed privacy-protected server-client interaction solution supports explicit knowledge generation for exploitation in federated learning. Extensive experimental results indicate that FedKIG significantly improves the generalisation performance and the stability of the model in heterogeneous federated learning scenarios.
Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that"increasing the number of global models yields better initialization,"and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.
This work introduces a novel multi-domain federated learning framework in which lightweight client-side proxy models collaborate with a server-side Foundation Model (FM) to learn new concepts without sharing private data.
Matteo Caligiuri, Francesco Barbato, Pietro Zanuttigh et al.· Trans. Mach. Learn. Res.· 0 citations
A domain-aware proxy selection framework to better adopt proxy data for OOD problems is proposed and the experimental results show that the proposed models effectively address the challenges of distribution shifts under OOD with and without proxy data.
This work proposes RAVEL-FCL, a generative replay-based framework for federated continual learning that integrates an improved generative model based on Rebooting ACGAN with multi-level feature alignment to ensure consistency and employs Elastic Variational Continual Learning on the server to probabilistically regularize the global model and preserve past knowledge.
Yurui Zhou, Jia Hu, G. Min et al.· ACM Transactions on Autonomo...· 0 citations
Federated Learning (FL) allows decentralized clients to train models collaboratively while preserving data privacy. However, distribution mismatch across clients often leads to poor global generalization and degraded local client-level performance. In such scenarios, some of the clients with their local models trained solely on local data may perform better than the globally learnt model, thus nullifying the benefits of collaborative federated learning. To address this, we propose SAPE-FL (Similarity-Aware Personalized Federated Learning), a novel personalization framework that anchors each client's model to both the global model and a similarity-weighted peer averaged model. By incorporating dynamic, client-specific regularization based on both model similarity and output similarity, SAPE-FL adaptively balances global knowledge transfer and peer collaboration while filtering out dissimilar clients. This dual anchoring mitigates negative transfer and enhances robustness in heterogeneous settings. We theoretically analyze our algorithm establishing its convergence guarantees and empirically show that SAPE-FL outperforms state-of-the-art methods under high statistical heterogeneity and low client data regimes.
A. Kumar, Sunil Gupta, Ngyuen Dang et al.· 0 citations
Federated training enables language models to learn from distributed private text, but the server cannot directly verify the local supervision or optimization process that produces each client update. A malicious client can therefore train on corrupted targets, introduce incorrect context-token associations, and degrade the global model through repeated aggregation. Such degradation can also increase the risk of unreliable or hallucinatory generation. We propose Federated Learning with Normalization Signatures (FedLNS), a server-side framework for lightweight malicious-update screening. FedLNS represents each client update through changes in trainable normalization-layer parameters and screens suspicious updates against a robust, history-aware cross-client reference. Because the signatures are extracted at the server from the returned local models, FedLNS requires no additional client-to-server parameter or metadata exchange compared to standard federated learning (FL) methods. After screening, the retained full-model updates can be aggregated using standard FL or another compatible aggregation rule. FedLNS requires no raw client data, trusted server dataset, labeled attack examples, or separately trained detector. Experiments on GPT-style, BERT-style, and LLaMA-style models trained from scratch with 200 clients show that, under 40% population-level target manipulation, FedLNS achieves lower test perplexity than the strongest of six baselines for all three architectures under both IID (independently and identically distributed) and non-IID data partitions.
Kai Li, Jong-Ik Park, Carlee Joe-Wong et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
