Ganqu Cui

SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning

This work introduces a student reference KL loss and mask the advantages of special termination tokens to mitigate the problem of excessive generation length and frequent truncation, and improves on science benchmarks such as HLE and HiPhO, suggesting that OPD transfers reasoning capabilities that generalize beyond the mathematical training domain.