Review Open access

Federated Learning with SVMs: Dynamic SGD for Efficient Hyperparameter Optimization

Deebakkarthi Chinnasame Rani Gowtham Ramesh Sountharrajan Sehar Varun Aiyaswamy Kannan Elambharathi Padmavathi Thangavel Bharath Kumar Kanapareddy

Aug 2026 · Journal of Computational and Cognitive Engineering · 0 citations · 27 references

TL;DR

Empirical evaluations on benchmark datasets show that federated SVM framework provides effective results comparable to NN-based FL approaches while significantly less computational and communication overhead.

Abstract

In the current digital landscape, organizations are actively looking for user data to make informed decisions at the edge. This raises a need for a solution that prioritizes user privacy while leveraging user's data. Federated learning (FL) emerges as a viable solution to address this issue. However, these methods are dominated by complex, resource-intensive neural networks (NN). This necessitates the development of an FL technique that protects user privacy while employing lightweight models. Support vector machines (SVMs) provide a lightweight alternative, but their traditional training techniques, such as quadratic programming and sequential minimal optimization, are inefficient and incompatible with FL. Existing stochastic gradient descent (SGD)-based SVM variants also depend on fixed or heuristic learning rates, which restrict convergence under non-IID data and client heterogeneity. In this paper, we propose a novel meta-learned dynamic learning rate controller for SGD-trained SVMs in federated settings. Unlike standard adaptive optimizers, the controller adapts learning rates over each epoch and clients. This process enables faster convergence and lower communication overhead to heterogeneous data. Empirical evaluations on benchmark datasets show that federated SVM framework provides effective results comparable to NN-based FL approaches while significantly less computational and communication overhead. Received: 29 September 2025 | Revised: 30 March 2026 | Accepted: 9 July 2026Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in Kaggle at https://git-disl.github.io/GTDLBench/datasets/mnist_datasets/ (MNIST), the NIST repository at https://www.nist.gov/itl/products-and-services/emnist-dataset (FEMNIST), and the University of Toronto's repository at https://www.cs.toronto.edu/~kriz/cifar.html (CIFAR-10). Author Contribution Statement Deebakkarthi Chinnasame Rani: Methodology, Software, Validation, Formal analysis, Investigation, Resources, Writing – original draft, Writing – review & editing. Gowtham Ramesh: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Writing – original draft, Writing – review & editing, Supervision, Project administration. Sountharrajan Sehar: Validation, Formal analysis, Investigation, Resources, Data curation, Writing – review & editing, Project administration. Varun Aiyaswamy Kannan: Formal analysis, Resources, Data curation. Elambharathi Padmavathi Thangavel: Formal analysis, Writing – original draft, Visualization. Bharath Kumar Kanapareddy: Investigation, Resources, Data curation.

DOI Semantic Scholar

Read PDF

Similar papers

Preprint Aug 2026

Global Federated Learning Strategies for Building Efficient Personalized Models

Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that"increasing the number of global models yields better initialization,"and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.

Seongyoon Kim · 0 citations

Preprint Aug 2026

Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning

Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space using an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client.

Mirko Konstantin, S. Zachow, Anirban Mukhopadhyay · 0 citations

Open access 2026

Federated Learning with Consistency Optimization Algorithms under Non-IID Data

This paper proposes Federated Learning with Consistency Optimization Algorithms (FedCO), a novel optimization framework that incorporates a label-skew-aware correction loss and neural feature distribution regularization during local training that significantly improves accuracy and convergence under diverse non-IID settings.

Ruiqi Wu, Yehong Li, Hongjie Guo et al. · Computers, Materials & C... · 0 citations

Open access Jul 2026

Optimization Stability in Federated Learning under Non-IID Data: A Comparative Study of FedAvg and FedProx with Deep Convolutional Neural Network Architectures

A thorough comparison between two well-known federated optimization algorithms, FedAvg and FedProx, and three popular deep convolutional neural network architectures such as ResNet18, VGG16 and VGG19 demonstrates that enforcing strong federated optimization coupled with fitting the appropriate deep convolutional architectures could provide a more reliable way of learning in decentralized settings.

Gitanjali Yadav, Jayashree V. Bagade · Journal of Intelligent Decis... · 0 citations

Book Open access Aug 2026

Optimal Metric-Aware Class Rebalancing in Federated Learning

This study proposes an analytical class rebalancing method to compute the optimal rebalancing ratios for imbalanced datasets, and is the first to provide an analytical and parameter-free solution to the problem.

Sihao Lv, Zian Le, Mingyang Zhou et al. · Proceedings of the 32nd ACM... · 0 citations

2025

Rising from Ashes: Generalized Federated Learning via Dynamic Parameter Reset

A novel FL framework is presented, FedPhoenix, that stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than speciﬁc overﬁtting features.

Jia-Hao Wu, Ming Hu, Yanxin Yang et al. · Neural Information Processin... · 1 citation