Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?
Since the advent of knowledge distillation, KL divergence has been the standard loss in distillation. Recently, on-policy distillation (OPD) has emerged as an efficient post-training paradigm for LLMs. As a distillation method, OPD naturally inherits KL divergence as its standard loss. However, in this work, we find th...
Wen-Ze Lin, Ji-Yuan Long, Jia-Le Zhao et al.
· 0 citations