Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation
A unified temporal-credit view of token-level OPD and sequence-level OPD is established, showing that practical token-level OPD can be interpreted as a temporal approximation to the sequence-level reverse-KL gradient.
Shi-Qi Liu, Ze-Yu He, Le-Tian Tao et al.
· 3 citations