Skip to content
Book Open access

FedTail-DT: A Dual-Teacher Framework for Long-Tailed Heterogeneous FL with CLIP Prototypes and Adaptive Aggregation

Aug 2026 · Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · pp. 1508-1517 · 0 citations · 14 references

Abstract

Federated learning enables decentralized model training while preserving data privacy, showing significant potential in sensitive fields such as healthcare and finance. However, real-world client data often exhibits both heterogeneity and long-tailed label distributions, leading to class bias and knowledge gaps that limit global model generalization. This paper proposes FedTail-DT, a novel approach for long-tailed heterogeneous federated learning, featuring three key innovations: (1) a dual-teacher–single-student architecture that integrates a pre-trained CLIP model as a semantic prior teacher and a globally aggregated model as an iterative optimization teacher to enhance tail-class perception; (2) CLIP text-prototype contrastive learning that uses text prototypes as semantic anchors to counteract feature bias caused by long-tailed distributions; (3) a multi-dimensional client scoring mechanism that dynamically calibrates aggregation weights based on data volume, category completeness, and update consistency. Experiments on federated benchmark datasets with various imbalances, for example, under extreme imbalance (factor 0.01), show that FedTail-DT outperforms existing SOTA methods by 4.21% on CIFAR-10-LT and 3.14% on CIFAR-100-LT, demonstrating its effectiveness in handling both data heterogeneity and long-tailed problems.

Read PDF

Similar papers

Preprint Aug 2026

Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning

Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space using an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client.

Mirko Konstantin, S. Zachow, Anirban Mukhopadhyay · 0 citations
Preprint Aug 2026

FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting

FedADB, a Class Anchor-Driven Dual-Branch FL framework, a dual-branch collaborative training mechanism designed for clients that achieves significant improvements in both accuracy and convergence speed.

Zhenyan Liu, Hua Zhang, Haoran Gao et al. · 0 citations
2025

A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments

This work proposes PHP-FL, a novel model-heterogeneous FL method that explicitly addresses scenarios with varying client participation probabilities to enhance both model accuracy and performance fairness, and introduces a Dual-End Aligned ensemble Learning (DEAL) module.

Siyuan Wu, Yongzhe Jia, Haolong Xiang et al. · 1 citation · ⚡1
#machine learning Preprint Sep 2026

Similarity-Aware Personalized Federated Learning in Heterogeneous Environments

Federated Learning (FL) allows decentralized clients to train models collaboratively while preserving data privacy. However, distribution mismatch across clients often leads to poor global generalization and degraded local client-level performance. In such scenarios, some of the clients with their local models trained solely on local data may perform better than the globally learnt model, thus nullifying the benefits of collaborative federated learning. To address this, we propose SAPE-FL (Similarity-Aware Personalized Federated Learning), a novel personalization framework that anchors each client's model to both the global model and a similarity-weighted peer averaged model. By incorporating dynamic, client-specific regularization based on both model similarity and output similarity, SAPE-FL adaptively balances global knowledge transfer and peer collaboration while filtering out dissimilar clients. This dual anchoring mitigates negative transfer and enhances robustness in heterogeneous settings. We theoretically analyze our algorithm establishing its convergence guarantees and empirically show that SAPE-FL outperforms state-of-the-art methods under high statistical heterogeneity and low client data regimes.

A. Kumar, Sunil Gupta, Ngyuen Dang et al. · 0 citations
Conference Aug 2026

FedLEAF: Bi-directional Prototype Adaptation for Heterogeneous Federated Learning

Prototype-based knowledge sharing effectively mitigates data and model heterogeneity in federated learning (FL) by exchanging class-level semantic information. However, existing methods typically assume all local prototypes are equally reliable. Consequently, low-quality prototypes from heterogeneous models or dynamic clients can contaminate the global aggregation, leading to a vicious cycle of noise accumulation and performance degradation. To address this, we propose FedLEAF, a Federated Learning framework with server-side proactive Evaluation and clientside Adaptive Fusion. Specifically, the server employs an Adaptive Learning Prototypes (ALP) network to dynamically evaluate prototype reliability and generate learnable aggregation weights, ensuring that highquality prototypes exert a primary influence on the global model. Meanwhile, the client utilizes a Historical Consistency Fusion (HCF) strategy to selectively absorb global knowledge by assessing its consistency with locally maintained historical prototypes. Extensive experiments on standard datasets demonstrate that FedLEAF achieves effective improvements in model accuracy and robustness compared to existing methods.

Zhiyuan Zhu, Si-Yi Deng, Dapeng Wu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.