#natural language process...
May 2026
MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation
Results show that self-contamination is a trainable component of the LiC gap, and propose MAIGO, an on-policy self-distillation method that reduces this contamination using history-cleaned references from the model's own policy.
Hao Zheng, Yun Zhu, Shurun Yuan et al.
· arXiv.org · 3 citations