UniSD integrates complementary mechanisms that address supervision reliability, representation alignment, and training stability, including multi-teacher agreement, EMA teacher stabilization, token-level contrastive learning, feature matching, and divergence clipping, and reveals when self-distillation improves over st...
Yiqiao Jin, Yi-Yang Wang, Lu-Cheng Fu et al.· arXiv.org· 7 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.