Skip to content

Joint Domain-Class Modeling for Federated Learning Under Feature Skew

Sep 2026 · arXiv.org · Vol abs/2609.22932 · 0 citations · 39 references
Computer Science

TL;DR

The proposed Joint Domain-Class Federated Learning (JDFL), a lightweight, optimizer-agnostic extension that makes this latent domain variation usable without sharing raw data, and introduces two complementary supervision strategies based on simple intuitions.

Abstract

Federated learning (FL) enables collaborative model training without centralizing private data, but performance often degrades under feature skew: clients share labels while the conditional input distributions $p_i(x\!\mid\!y)$ vary due to latent, client-specific appearance factors. We propose Joint Domain-Class Federated Learning (JDFL), a lightweight, optimizer-agnostic extension that makes this latent domain variation usable without sharing raw data. JDFL first infers domain clusters called pseudo-domains from brief local update signals. It then expands the classifier head to output $M\times C$, joint (domain-class) logits. This allows the model to represent domain-conditioned appearance while keeping a shared backbone. To train the expanded head we introduce two complementary supervision strategies based on simple intuitions: a similarity-aware soft-labeling that transfers evidence between nearby inferred domains while allowing domain-specific specialization, and a per-sample randomized target assignment that perturbs supervision across the joint outputs and serves as a low-cost training-time regularizer. JDFL integrates with existing standard FL methods (e.g., FedAvg, SCAFFOLD) with minimal changes. Empirically, both supervision modes consistently improve global test accuracy on standard domain-shifted image benchmarks; ablations and sensitivity studies show the gains stem from the proposed supervision and parametrization rather than mere capacity increase.

View source

Similar papers

Open access 2026

FedProRef: Communication-Efficient Prototype-Count Supervision for Federated Learning Under Severe Label Skew

Severe label skew leaves local classifier heads with insufficient supervision for underrepresented classes in federated learning (FL). We present FedProRef, an upload-efficient prototype-count supervision framework for frozen-feature, head-only FL. Each eligible client–class pair uploads only a prototype and sample cou...

Lei Wang, Hao Ren · 0 citations
Preprint Aug 2026

FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learning

FedCC, a simple and effective algorithm for mitigating client misclassification that balances confidence in majority classes against uncertainty in under-represented ones, is proposed and significantly outperforms existing methods.

Wenxuan Ye, Onur Ayan, Xue-Li An et al. · 0 citations
2026

Federated Generalized Category Discovery via Personalized Contrastive Graph Learning

In this work, we investigate the task of Federated Generalized Category Discovery (Fed-GCD), which aims to leverage labeled data from known classes to cluster unlabeled samples from both known and unknown classes through privacy-preserving collaboration among clients. Existing Fed-GCD methods predominantly emphasize im...

Wen-Jing Li, Nan Pu, Jessica Fan et al. · 0 citations
Aug 2026

When Federated Prompt Learning Fails: Understanding and Mitigating Cross-Domain Prompt Interference.

Federated prompt learning (FPL) offers a parameter-efficient way to adapt large-scale vision-language models by collaboratively optimizing lightweight prompts across distributed clients. Existing approaches typically assume that prompts learned on heterogeneous domains can be safely aggregated. We show that this assump...

Yun-Feng Zhao, Jun Wang, Dong Wu et al. · 0 citations
#machine learning Preprint Sep 2026

Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation

Component-wise diagnostics and convergence results support the role of persistent heads and distillation-guided local optimization under label-skewed data and suggest that pFedKDH obtains the best accuracy in most settings.

Polycarpo Souza, José Mairton Barros da Silva, C. Cavalcante · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.