Towards Neuro-Symbolic Federated Tree Learning

Global Federated Learning Strategies for Building Efficient Personalized Models

Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that"increasing the number of global models yields better initialization,"and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.

Rising from Ashes: Generalized Federated Learning via Dynamic Parameter Reset

A novel FL framework is presented, FedPhoenix, that stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features.

Optimization Stability in Federated Learning under Non-IID Data: A Comparative Study of FedAvg and FedProx with Deep Convolutional Neural Network Architectures

Federated learning (FL) has emerged as a prospective model training paradigm for distributed clients, which provides data privacy while maintaining the ability to train a collaborative model.In healthcare, finance and edge intelligence applications, federated learning (FL) is a promising model training paradigm for distributed clients, which preserve data privacy while retaining the capacity of collaboratively training a model. But, this optimization when data is not independent and identically distributed (non-IID), is still a great challenge because of the heterogeneity of clients and unstable model convergence. In this paper, we provide a thorough comparison between two well-known federated optimization algorithms, FedAvg and FedProx, and three popular deep convolutional neural network (DCNN) architectures such as ResNet18, VGG16 and VGG19. This study comprehensively assesses these models in a 10-client federated setting where data is partitioned into class-skewed non-IID data sets, by examining optimized stability, convergence and classification performance across 20 communication rounds. The experimental setup investigates how the proximal regularization can reduce the client drift and enhance the learning uniformity of the client datasets. The experimental results show that FedProx consistently shows better convergence stability and better predictive performance than FedAvg as the data becomes more and more heterogeneous, whereas ResNet18 provides a good trade-off between the model complexity, communication efficiency and classification performance. The results demonstrate that enforcing strong federated optimization coupled with fitting the appropriate deep convolutional architectures could provide a more reliable way of learning in decentralized settings. This work offers practical learnings for designing scalable, privacy-preserving and stable FL systems in real-world non-IID applications.

Federated Learning with Differential Privacy: A Comprehensive Framework for Privacy-Preserving Distributed Machine Learning

Overview: Federated Learning (FL) is a type of machine learning in which multiple clients use their own data to train a model. This does not require sending the raw data from each client to a central server. However, FL is currently very vulnerable to several types of attacks, including inference, model inversion, and membership inference. In this study, we have implemented a comprehensive experimental framework for analysing FL performance using standard FL aggregation protocols FedAvg, FedProx, and SCAFFOLD in conjunction with Differential Privacy (DP) mechanisms; specifically, the Gaussian noise mechanism with Rényi Differential Privacy (RDP) accountants. Our experiments all used K = 100 simulated clients, with heterogeneous data distribution across the entire federation (non-IID), and the respective clean datasets used were MNIST and CIFAR-10. For example, in our IID scenarios, our DP-FedAvg protocol achieved an accuracy of 84.9% on MNIST with ε = 1.0 and δ = 10^-5 (i.e., no privacy guarantees and 92.6% reuse of the same parameters). This corresponds to a 7.7-percentage-point trade-off in accuracy for provable privacy. In our non-IID scenario, we achieved an accuracy of 72.8% for the exact same privacy parameters and an LDA αᴸᴸᴸ = 0.5. We systematically analyse how various combinations of clipping norms on gradients, noise multipliers, and client participation rates converged. All results have been reported as-is, along with the associated standard deviation across each of the five independent sample runs that we performed using different random seed values. Finally, our framework will assist practitioners deploying privacy-friendly FL within either healthcare, banking, or Internet of Things environments.

Personalized federated learning based on self-excluding aggregation with knowledge distillation

Domain Sensitive Federated Learning with Fisher-Informed Pruning