Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that"increasing the number of global models yields better initialization,"and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.
Federated learning (FL) enables collaborative model training across distributed clients while keeping data local. A central challenge is determining which client updates are beneficial for aggregation with respect to each client's target domain. Existing methods typically address this problem in parameter space by comparing model parameters or gradients. However, parameter-space similarity can be a poor proxy for predictive behavior, especially under heterogeneous, non-IID data. Consequently, updates that are misaligned with a client's target domain, including those caused by heterogeneous data or malfunctioning clients, may degrade local model performance. We propose Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space. LIGHTYEAR uses an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client. By relating model parameters to local predictive responses, the Neural Tangent Kernel (NTK) provides a more expressive criterion for update selection than parameter-space similarity alone. Because function-space information is not available before aggregation in conventional centralized FL, LIGHTYEAR uses a peer-to-peer (P2P) topology in which clients exchange updates directly and evaluate incoming models on private validation data. Each client selects only updates that are beneficial for its own target domain and aggregates them using a regularized rule that improves stability under heterogeneity. Across five datasets and nine baseline methods, LIGHTYEAR consistently outperforms centralized FL baselines and existing P2P approaches.
Mirko Konstantin, S. Zachow, Anirban Mukhopadhyay· 0 citations
A novel FL framework is presented, FedPhoenix, that stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features.
Jiahao Wu, Ming Hu, Yanxin Yang et al.· Advances in Neural Informati...· 1 citation
To address the conflict between model generalization and personalized recommendation in distributed educational environments, this paper constructs a personalized learning path recommendation model based on federated learning. Such privacy-preserving collaborative learning frameworks are also valuable for intelligent information processing and distributed decision-making in modern electromagnetic communication and edge computing systems, where data sharing is often restricted. The proposed model employs a Graph Neural Network (GNN)-Transformer hybrid encoder that combines knowledge graphs with learning behavior sequences to accurately capture knowledge transfer relationships. A dynamic knowledge distillation aggregation strategy is introduced to generate soft labels from the global model for guiding local optimization, thereby preserving personalized characteristics while improving semantic consistency. Furthermore, an adaptive aggregation mechanism based on Kullback-Leibler (KL) divergence dynamically adjusts client weights to enhance robustness under heterogeneous data distributions. Experimental results demonstrate that the proposed method achieves excellent recommendation accuracy (average Hit@5 of 0.676 ± 0.009), sequence consistency (average NDCG@10 of 0.712 ± 0.008), and personalized responsiveness (average personalized score difference rate of 0. 38). The framework effectively balances global generalization and local adaptation while maintaining privacy protection, providing a feasible solution for secure and intelligent recommendation in distributed learning environments and offering technical insights for collaborative intelligence in privacy-sensitive electromagnetic information systems.
The federated learning (FL) paradigm fosters distributed pervasive computing combined with artificial intelligence techniques, allowing for optimized data usage and improved mitigation of privacy concerns. Indeed, model training occurs on the client's local devices, and model parameters are subsequently shared with a centralized server. However, there is a need to find a tradeoff between models'personalization and generalization capabilities. In this paper, we design and implement several testing scenarios devoted to evaluating and comparing the centralized, local, and federated paradigm performances. We also design and implement a scenario that emulates a change in clients'data. We then present experimental results of the FedAvg algorithm applied to the Human Activity Recognition (HAR) domain to understand the trade-off between personalized and generalized accuracy. Results show that, although FedAvg confirms a higher degree of personalization capabilities while keeping a high degree of generalization with respect to the traditional centralized learning, this result is not so obvious under stressful conditions, such as when varying class distribution over clients.
Andrea Luna, Susanna Peretti, C. Contoli et al.· 0 citations