On-policy distillation (OPD) supervises student-generated responses using sampled-token student-teacher log-ratios. A positive log-ratio penalizes a token that the student assigns higher probability to than the teacher, but its current value does not distinguish sustained high penalties from abrupt increases along a re...
Guang-Yu Liu, Si-Yuan Gan, Shi-Long Zhou et al.· 0 citations
Direct Advantage Amplification (DAA), which amplifies the advantages of hard-to-sample correct responses on hard prompts, as obtained by Dynamic Sampling, is proposed, which ensures that, when Dynamic Sampling is used, these hard-to-sample responses can be effectively capitalized on, implying higher training efficiency...
Si-Yuan Gan, Yu-Hang Li, Xi-Ran Wang et al.· 0 citations
RA-OPD selects more reliable trajectories to improve student model performance without requiring additional computational cost and is evaluated on math and code benchmarks using models from the Qwen3 family and the DeepSeek-R1 family.
Si-Yuan Gan, Yu-Hang Li, Xi-Ran Wang et al.· 2 citations· ⚡1
This work advocates for Joint Online-Offline Fine-Tuning as a superior paradigm that breaks the convention of restricting offline data to SFT and online data to RFT, and provides the first comprehensive survey focusing specifically on the synchronization of data provenance.
Tai-Hang Zhen, Guang Yang, Chen-Zhang Li et al.· Proceedings of the Thirty-Fi...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.