Boosting Gradient-Based Training Diagnosis for Efficient and Accurate Federated Learning

Global Federated Learning Strategies for Building Efficient Personalized Models

Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that"increasing the number of global models yields better initialization,"and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.

FedLDCS: Adaptive Divergence-Based Client Selection for Federated Learning

Towards Neuro-Symbolic Federated Tree Learning

Federated Learning (FL) is a distributed machine learning paradigm that enables multiple clients to collaboratively train a model while keeping their data localized, thus preserving privacy. While most FL approaches focus on deep learning models, there is a growing interest in exploring alternative models that can offer interpretability and efficiency. This paper presents TreeClimber, a novel neuro-symbolic approach for training decision tree models in a federated setting using gradient-free optimization techniques. TreeClimber embeds data into a subsymbolic latent space via a randomized embedding, from which decision trees are reconstructed through a neural architecture. Trees are optimized in the embedding space in a federated setting using a zeroth-order approximation of the gradient, allowing for effective training of interpretable decision models without direct access to data. Experimental results demonstrate the effectiveness of TreeClimber in enabling secure distributed training of decision trees without sharing raw information about local datasets, achieving competitive performance compared to traditional centralized training methods.

Rising from Ashes: Generalized Federated Learning via Dynamic Parameter Reset

Although Federated Learning (FL) is promising for privacy-preserving collaborative model training, it suffers from low inference performance due to heterogeneous client data. Due to heterogeneous data across clients, FL training easily learns client-specific overfitting features. Existing FL methods adopt coarse-grained averaging, which can easily cause the global model to get stuck in lo-cal optima, leading to poor generalization. Specifically, this paper presents a novel FL framework, FedPhoenix, to address this issue. It stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features. Experimental results on various well-known datasets demonstrate that compared to SOTA FL methods, FedPhoenix can achieve up to 20.73% higher accuracy. The implementation is publicly available at https://github.com/UniString/FedPhoenix .

HCFL: hybrid contribution-driven federated learning for fair and efficient optimization.

Federated Learning (FL) enables collaborative model training across decentralized data silos while preserving data privacy. However, client selection strategies in conventional FL processes typically rely on single-dimensional evaluation metrics, which fail to capture data diversity and overlook the dynamic nature of client contributions, particularly in domains characterized by sparse and heterogeneous data, such as healthcare and drug discovery. These limitations ultimately hinder the global model's generalization ability and reduce training efficiency. To address these challenges, this paper proposes an adaptive FL framework that employs a hybrid contribution evaluation mechanism as the core principle for client selection and resource management. The proposed approach quantifies each client's effectiveness by integrating two complementary dimensions: (i) a performance-based evaluation that measures the immediate impact of a client's update on the global optimization trajectory, and (ii) a coverage-based evaluation that estimates data diversity in the latent embedding space without exposing raw data. By combining these two criteria, the hybrid mechanism ensures that highly contributive clients are preferentially selected while preventing the permanent exclusion of any participant, thereby maintaining a balanced trade-off between efficiency and fairness. Experimental results demonstrate that the proposed framework outperforms existing FL baselines in terms of training efficiency, data utilization, and fairness.

Federated learning for medical image classification based on prototype alignment