1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillation
An information-efficiency ratio (IER) based on a signal-to-noise decomposition and a candidate-set approximation enables token selection based on IER and its combination with existing usefulness scores, while retaining the sampled reverse-KL training objective.