FedOPAL: One-Shot Federated Learning via Analytic Visual Prompt Tuning

Federated Lightweight Fine-Tuning

Federated fine-tuning is bottlenecked by communication: FedAvg and pseudo-gradient schemes transmit a payload that scales with the model, and gradient compression shrinks it by only a constant factor. We take a different lever. Mapping networks generate a network's weights from a small trainable latent through a frozen affine projection; because the map is shared and affine, averaging latents is exactly averaging the generated weights. We turn this into a practical low-bandwidth federated channel with two changes: a low-rank, seed-regenerable factorisation of the projection (cutting generator memory from ~80 GB to ~10 MB), and a delta formulation $\theta = \theta^{\mathrm{pre}} + U V^{\top} z$ that learns an additive correction around a shared centrally-pretrained base -- federated fine-tuning, which is what makes the method work at scale. A frozen orthogonal classifier head further removes the head from the payload while improving accuracy. On CIFAR-100 with ResNet-18+GroupNorm, our method (FLITE, Federated Low-rank Iterative Training Engine) communicates 1,280 floats (~5 KB) per client per round -- an 8718x reduction -- and reaches 74.67%, within ~0.5 pp of full-weight FedAvg. The averaging identity holds to floating-point precision ($6 \times 10^{-8}$); the method sits one to two orders of magnitude below PowerSGD and top-k on the bandwidth-accuracy Pareto; it matches or exceeds full-weight FedAvg under strong non-IID skew. int4 latents reach 648 bytes per round at unchanged accuracy, whereas int4 full-weight FedAvg collapses to chance.

One-Shot Federated Learning With Lightweight Intermediate Models in Wireless Sensor Network Setting

One-shot federated learning (FL) completes model training and aggregation in a single communication round, significantly reducing communication costs compared to traditional FL. This approach is particularly suitable for resource-constrained environments such as wireless sensor networks (WSNs). However, existing solutions face significant challenges in aggregation owing to model heterogeneity, where clients adopt architectures of varying depth, width, and computational capacity. To address this issue, we propose a one-shot FL method named FedLIM, which employs a lightweight intermediate model for efficient knowledge transfer and global model aggregation. The Fisher information matrix (FIM) is incorporated to guide the model aggregation process and improve its robustness. Although FedLIM completes global training and aggregation in a single communication round, an optional personalized model adjustment step is introduced afterward. This step only involves server-to-client distribution without additional aggregation. Experimental results on three datasets demonstrate that FedLIM achieves superior global model accuracy compared to existing one-shot FL methods, particularly in highly heterogeneous environments. Moreover, the accuracy of local models is further enhanced through this optional refinement step.

Regularization-Enhanced Federated Class-Incremental Learning with Generative Replay for Smart Edge

Federated learning is an emerging paradigm that addresses privacy concerns by training on decentralized data, but it faces significant performance challenges due to factors like non-independent and identically distributed (non-IID) data and dynamic data distributions. These challenges are further amplified in federated class-incremental learning under realistic, non-monotonic class streams, where previously seen classes can intermittently reappear alongside new classes—making generative replay highly susceptible to training instability, mode collapse, and feature drift. To address these issues, we propose RAVEL-FCL, a generative replay-based framework for federated continual learning. Our approach integrates an improved generative model based on Rebooting ACGAN with multi-level feature alignment to ensure consistency, and employs Elastic Variational Continual Learning on the server to probabilistically regularize the global model and preserve past knowledge. Experimental evaluations show that our method achieves stable convergence and significantly outperforms state-of-the-art methods on standard benchmarks—for example, it improves accuracy by 5.27% over FedCIL on EMNIST-Balanced and by 1.60% and 1.27% over the strongest baselines on TinyImageNet and ImageNet-LT, respectively—while maintaining a forgetting rate below 5% across tasks. By jointly addressing feature drift and catastrophic forgetting, RAVEL-FCL provides a robust and privacy-preserving solution suitable for the adaptive and autonomous systems envisioned for Industry 5.0.

Prompt transfer for few-shot fault diagnosis in IIoT: a federated learning framework with tabular foundation models

In industrial internet of things (IIoT) deployments, condition monitoring data is rarely complete. Labeled samples are scarce, sensors drop out, signals are corrupted by noise, and what reaches the edge is often a sparse, distorted fragment of the ideal dataset. Federated learning preserves privacy, yet its standard formulations assume clients can train locally on adequate labeled data-an assumption that seldom holds on the factory floor. This paper proposes prompt transfer, a distinct federated learning paradigm that replaces conventional gradient or prototype exchange with the transfer of small, curated sets of labeled examples (prompts) as the sole medium of knowledge flow. Unlike federated distillation, which shares model outputs, or meta-learning, which requires local gradient adaptation, prompt transfer leverages the in-context learning capability of a frozen tabular foundation model to achieve zero-gradient, parameter-free knowledge transfer across clients. Experiments on the Southeast University bearing dataset show that under 5-shot settings, our method FedICL achieves 94.0% accuracy, in contrast to FedAvg’s random-guessing performance of 20.1% while requiring zero gradient computation and reducing communication rounds by 95% and data transfer by 87%. The approach remains effective under non-IID distributions ( 53.9% vs 20% random). This work directly addresses the special issue themes of incomplete data and federated learning, offering a practical solution for label-scarce, resource-constrained IIoT scenarios.

Domain Sensitive Federated Learning with Fisher-Informed Pruning

Scaling Synthetic-Image Pre-Training for Federated Fine-Tuning of Large Vision Models

Federated fine-tuning (FedFT) enables adapting pre-trained large vision models (LVMs) on distributed, privacy-sensitive devices, while its practical deployment is hindered by three critical challenges: resource constraints, system heterogeneity, and non-IID data. While prior studies partially address these issues, e.g., by pre-training initial models on synthetic images to mitigate the adverse effects of non-IID data, or leveraging parameter-efficient fine-tuning (PEFT) methods like low-rank adaptation (LoRA) to reduce resource consumption, they remain inadequate and fragmented. Specifically, existing synthetic image generation methods fail to capture device-specific feature distributions, while current PEFT-based FedFT methods often undervalue weaker devices that may provide critical information. More importantly, the separate optimization of pre-training and FedFT neglects their inherent connection, lacking a holistic perspective to maximize training efficiency. To overcome these limitations, we propose FeDiSyn, a unified framework that holistically considers the interplay between pre-training and FedFT to minimize the overall LVM training time. Specifically, FeDiSyn introduces: (i) a scaling law for FedFT pre-training to determine the optimal number of synthetic images, balancing pre-training benefit against generation/pre-training cost, (ii) diffusion-based synthetic image generation that captures device-specific feature distributions for pre-training to tackle non-IID data, and (iii) a contribution-aware LoRA configuration and bandwidth allocation algorithm for FedFT to ensure that informative devices are effectively utilized while addressing system heterogeneity. Experimental results on the real-world testbed demonstrate that FeDiSyn reduces completion time by over 52.5% and communication cost by over 97.2%, while achieving comparable accuracy to state-of-the-art solutions.