Preprint
Aug 2026
Mismatch Matters: On-Policy Distillation Beyond Token Agreement
TIDE (Token-level Independent Deficit-Excess correction), which applies bounded Hellinger shaping to suppress the most severe sampled excesses and an analytic teacher top-$K$ injection to restore deficient probability mass without requiring deficit tokens to be sampled, is proposed.
Zi-Chao Yu, Chengzhi Yu, Sheng-Ze Xu et al.
· 4 citations