FedML-DKD is proposed, a robust framework that integrates adaptive Decoupled Knowledge Distillation with federated meta-learning and introduces a heterogeneity-aware weighting mechanism based on Earth Mover’s Distance that helps prevent overfitting to biased local data while improving the reliability of knowledge transfer.
It is demonstrated that prediction-space meta-learning constitutes a practical alternative for federated aggregation without requiring shared parameter initialization, and among the different forms of heterogeneity in federated learning, this work focuses on statistical and model heterogeneity.
This work proposes Class-wise Reliability-Aware Distillation (CRAD), which, per class, first discards teachers that disagree with the peer consensus and then takes a weighted average of the rest, weighting each teacher by its per-class reliability (precision, or inverse variance).
Baraa Bilbeisi, Meng-Chen Fan, Bao-Cheng Geng et al.· 1 citation
This study delves into large PFMs adaptation in the resource-constrained federated learning environment, and proposes an innovative framework, namely ADRAP, which alternates between large model distillation and resource-adaptive pruning, with guaranteed convergence.
Xiao Zhang, Yang-Yang Wang, Xing-Yu Sun et al.· IEEE Transactions on Pattern...· 0 citations
A heterogeneous compression framework for FedKD is proposed that enables each client to select a compression strategy from a candidate strategy set, and an Adaptive heterogeneouS Compression algorithm for fEderated kNowledge Distillation (ASCEND), which employs an exponential moving average (EMA)-enhanced $\epsilon$-gr...
Chen-Wang Liu, Yijun Liu, Chang Liu et al.· 0 citations
Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. Thi...
This work proposes SAPE-FL (Similarity-Aware Personalized Federated Learning), a novel personalization framework that anchors each client's model to both the global model and a similarity-weighted peer averaged model that mitigates negative transfer and enhances robustness in heterogeneous settings.
V. ArunKumarA, Sunil Gupta, N. Dang et al.· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 9, 2026
A machine-learning approach called CheMeleon provides a practical way to develop new drugs or other molecules with much less detailed, expensive experimentation.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.