TL;DR
This work proposes three curriculum variants on top of a RLOO fine-tuning baseline that can concentrate RLOO training on this frontier and improve final performance on the Countdown arithmetic reasoning task.
Desc descriptive evidence is provided that long-horizon multi-tool post-training can change ways of working that transfer beyond its training domain, and both software-engineering benchmarks improve despite the training collection containing no software-engineering tasks.
Best Practice Critic Optimization (BPCO) is developed, a recipe that combines DPPO, value predictions bounded to the reward range, Monte Carlo value targets, unnormalized policy advantages, and length-adaptive generalized advantage estimation and shows that a carefully designed critic provides a reliable alternative to...
AdaKP is an online selector that re-chooses each problem's KP subset over the course of RL training, an entropy proxy that scores a KP by the reduction in next-token entropy it induces in a single inexpensive forward pass, with a provable bound on its truncation bias.
Task Specialization Fine-Tuning (TSFT), an online framework that predicts fine-tuning performance with a simple parametric model and exactly solves the resulting discrete budget allocation problem via integer linear programming, is proposed.
LLM-enhanced automatic curriculum learning (LEACL), a framework that integrates LLMs and ACL to address limitations of ACL, is proposed, which achieves better asymptotic performance in terms of the success rates compared to human-designed dense rewards.
This work proposes a self-evolving curriculum that, each round, evaluates the current checkpoint, seeds QbQ from the problems it can mostly get right, and trains on the resulting variants; under an identical data budget, this breaks the ceiling and lifts pass@1 to 16.5\% with no sign of saturation after 20 rounds.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
