Skip to content
Conference

Personalized Federated Learning via Double Knowledge Distillation for Heterogeneous Model Deployments

Aug 2026 · 2026 IEEE/CIC International Conference on Communications in China (ICCC) · pp. 414-418 · 0 citations · 15 references

Abstract

Due to distributed data and privacy concerns, federated learning(FL) is a promising approach to learn global models from distributed data, with personalized federated learning (PFL) being a key enabler for customized services in future 6G networks. Federated Distillation (FD) is a classic communication-efficient PFL paradigm. It uses knowledge distillation to learn the strategy of model personalization. Although the original FD algorithm, also known as FedDistill, theoretically supports the direct deployment of heterogeneous model structures across clients, it can degrade the accuracy performance. To address this, we propose a new PFL method, Personalized Federated Double Knowledge Distillation (pFedDKD), that exploits knowledge distillation in both the global aggregation and local update phases. Moreover, we develop a variant, pFedDKD-SSBLA, that adaptively aggregates the highest-quality logits generated across different model depths. The experiment shows that the proposed pFedDKD and pFedDKD-SSBLA not only show better performance on average accuracy than FedDistill with heterogeneous models deployed, but also consistently outperform the state-of-the-art (SOTA) baseline, FedProto.

View source

Similar papers

EMD-guided adaptive decoupled knowledge distillation for federated meta-learning under data heterogeneity

FedML-DKD is proposed, a robust framework that integrates adaptive Decoupled Knowledge Distillation with federated meta-learning and introduces a heterogeneity-aware weighting mechanism based on Earth Mover’s Distance that helps prevent overfitting to biased local data while improving the reliability of knowledge trans...

De-Bao Wang, Shao-Peng Guan · 0 citations
#machine learning Preprint Sep 2026

Similarity-Aware Personalized Federated Learning in Heterogeneous Environments

This work proposes SAPE-FL (Similarity-Aware Personalized Federated Learning), a novel personalization framework that anchors each client's model to both the global model and a similarity-weighted peer averaged model that mitigates negative transfer and enhances robustness in heterogeneous settings.

V. ArunKumarA, Sunil Gupta, N. Dang et al. · 0 citations
Preprint Aug 2026

Out-of-Distribution Federated Distillation with Domain-Aware Proxy

A domain-aware proxy selection framework to better adopt proxy data for OOD problems is proposed and the experimental results show that the proposed models effectively address the challenges of distribution shifts under OOD with and without proxy data.

Jiahao Xiao, Jiangming Liu · 0 citations
Preprint Aug 2026

Global Federated Learning Strategies for Building Efficient Personalized Models

Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. Thi...

Seongyoon Kim · 0 citations
Preprint Aug 2026

Adaptive Heterogeneous Compression for Resource-Efficient Federated Knowledge Distillation

A heterogeneous compression framework for FedKD is proposed that enables each client to select a compression strategy from a candidate strategy set, and an Adaptive heterogeneouS Compression algorithm for fEderated kNowledge Distillation (ASCEND), which employs an exponential moving average (EMA)-enhanced $\epsilon$-gr...

Chen-Wang Liu, Yijun Liu, Chang Liu et al. · 0 citations
Conference Open access Sep 2026

Revisiting Hypernetwork in Model Heterogeneous Personalized Federated Learning

This work revisits hypernetworks and proposes a model heterogeneous personalized federated learning framework, MH-pFedHN, which quantifies clients with different architectures using customized embedding vectors and then generates client-specific model parameters through a server-side hypernetwork.

Chen Zhang, Husheng Li, Xiang Liu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.