MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation
This work proposes MI-Distillation, a framework that constructs a continuous Instruct-Reasoning data spectrum through model interpolation, and introduces SeqLSS, which favors reasoning paths that are both informative and learnable for the student.
Yangsong Lan, Ren-Kai Hu, Hong-Kai Zheng et al.
· 0 citations