Preprint
Aug 2026
Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models
It is found that OPD transfers a teacher's reasoning behavior rather than its answers to particular problems: training difficulty barely matters, and even problems the teacher never solves are useful.
Zhaoyi Li, Deyang Kong, Yuan Wei et al.
· 0 citations