This work systematically evaluates 16 PI configurations across three multi-turn agentic benchmarks and three student models that vary in scale and reasoning mode and finds that PI effectiveness depends on its content, generation source, and the student model.
Kai-Jian Zou, Emiliano Penaloza, MohammadHossein Rezaei et al.· 0 citations
Long-horizon, semi-verifiable agentic tasks often rely on rubric-based evaluation, but the evidence needed to assess these tasks can be spread across the environment state and the execution trace. A judge may overlook this evidence or accept unsupported claims, including an agent's own reports of success. When rubric v...
Reza Sanayei, Razvan-Gabriel Dumitru, Sun-Jie Hou et al.· 0 citations
RSI-Forge is introduced, a multi-agent pipeline that turns published papers into executable environments for self-improvement and provides a scalable approach to constructing research environments for training and evaluating self-improving agents.
Ren-Xiong Wang, Darvin Yi, Abril Herrlein et al.· 2 citations
CRAFT is introduced, a method that converts any rubric based evaluation dataset into a model specific diagnosis of weak capabilities, and yields both a sharper picture of what a model cannot do and measurably better models after finetuning on that diagnosis.
Vipul Gupta, Zihao Wang, Razvan-Gabriel Dumitru et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.