MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation
This work proposes ExpertAlign, a framework that routes supervision over the full teacher pool at each token, without domain labels or training a separate routing model, and demonstrates token-level routing can exploit cross-domain complementary supervision, and reduce exclusive reliance on prompt-level domain assignme...
Tian-Ze Xu, Yan-Zhao Zheng, Zhen-Tao Zhang et al.
· 0 citations