On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at every state, but nearby settings may offer additional supervision. We find that local parameter per...
Xin Wei, Yi-Fan Ding, Yoshua Y. Li et al.· 0 citations
A critique-in-the-loop self-improvement method that incorporates critique-based supervision into the actor’s self-training process and improves the actor’s exploration efficiency and solution diversity, especially on challenging queries, leading to a stronger actor model.
Zhi-Heng Xi, Dingwen Yang, Jixuan Huang et al.· Proceedings of the Thirty-Fi...· 0 citations
SA is proposed, a Stable Advantage Fusion framework that avoids entropy collapse and consistently outperforms fixed-coefficient GRPO+OPD fusion, improving the aggregate score by 0.70% across all six model-domain settings while achieving more stable training.
Yifan Ding, Xin Wei, Yoshua Y. Li et al.· arXiv.org· 3 citations
DagEvo is introduced, whose diagnostician extracts recurring error causes from this history and stores them in a hierarchical error-cause memory, whose diagnostician extracts recurring error causes from this history and stores them in a hierarchical error-cause memory.
Xin Wei, Yinze Ding, Yoshua Y. Li et al.· 0 citations
This work eliminates external gold supervision from the RL training reward: the model's own evaluative capability generates learning signals for its optimization -- a closed-loop setting of bounded recursive self-improvement (RSI) termed Recursive Self-Evaluation (RecurSE).
Kai-Yuan Liu, Ziyuan Zhuang, Rong-Xiang Weng et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.