Reinforcing What the Student Has Learned: Teacher-Guided On-Policy Distillation
Student-Delta On-Policy Distillation (SD-OPD), which uses centered student--base differences as update coefficients, keeps each coefficient only when its sign agrees with centered teacher--student feedback, and recomputes the student signal every training round.
Bo-Zhen Fang, Lin-Hao Zhong, Xue-Hao Wang et al.
· 0 citations