Skip to content

EMD-guided adaptive decoupled knowledge distillation for federated meta-learning under data heterogeneity

Sep 2026 · Cluster Computing · Vol 29 · 0 citations · 43 references
Computer Science

TL;DR

FedML-DKD is proposed, a robust framework that integrates adaptive Decoupled Knowledge Distillation with federated meta-learning and introduces a heterogeneity-aware weighting mechanism based on Earth Mover’s Distance that helps prevent overfitting to biased local data while improving the reliability of knowledge transfer.

View source

Similar papers

Open access Aug 2026

A Meta-Learning-Based Aggregation Strategy for Heterogeneous Federated Learning Scenarios

It is demonstrated that prediction-space meta-learning constitutes a practical alternative for federated aggregation without requiring shared parameter initialization, and among the different forms of heterogeneity in federated learning, this work focuses on statistical and model heterogeneity.

Sergio Pérez-Picazo, Hiram Galeana-Zapién, Edwin Aldana-Bobadilla · 0 citations
#machine learning Preprint Aug 2026

CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learning

This work proposes Class-wise Reliability-Aware Distillation (CRAD), which, per class, first discards teachers that disagree with the peer consensus and then takes a weighted average of the rest, weighting each teacher by its per-class reliability (precision, or inverse variance).

Baraa Bilbeisi, Meng-Chen Fan, Bao-Cheng Geng et al. · 1 citation
Sep 2026

Alternating Distillation and Resource-Adaptive Pruning for Federated Large Model Adaptation.

This study delves into large PFMs adaptation in the resource-constrained federated learning environment, and proposes an innovative framework, namely ADRAP, which alternates between large model distillation and resource-adaptive pruning, with guaranteed convergence.

Xiao Zhang, Yang-Yang Wang, Xing-Yu Sun et al. · 0 citations
Preprint Aug 2026

Adaptive Heterogeneous Compression for Resource-Efficient Federated Knowledge Distillation

A heterogeneous compression framework for FedKD is proposed that enables each client to select a compression strategy from a candidate strategy set, and an Adaptive heterogeneouS Compression algorithm for fEderated kNowledge Distillation (ASCEND), which employs an exponential moving average (EMA)-enhanced $\epsilon$-gr...

Chen-Wang Liu, Yijun Liu, Chang Liu et al. · 0 citations
Preprint Aug 2026

Global Federated Learning Strategies for Building Efficient Personalized Models

Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. Thi...

Seongyoon Kim · 0 citations
#machine learning Preprint Sep 2026

Similarity-Aware Personalized Federated Learning in Heterogeneous Environments

This work proposes SAPE-FL (Similarity-Aware Personalized Federated Learning), a novel personalization framework that anchors each client's model to both the global model and a similarity-weighted peer averaged model that mitigates negative transfer and enhances robustness in heterogeneous settings.

V. ArunKumarA, Sunil Gupta, N. Dang et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.