Long-horizon, semi-verifiable agentic tasks often rely on rubric-based evaluation, but the evidence needed to assess these tasks can be spread across the environment state and the execution trace. A judge may overlook this evidence or accept unsupported claims, including an agent's own reports of success. When rubric v...
Reza Sanayei, Razvan-Gabriel Dumitru, Sun-Jie Hou et al.· 0 citations
Agentic benchmarks guide model selection and training. Yet an agent can pass a task without demonstrating the intended capability. Such outcomes constitute unearned passes; their proportion among all passes defines the integrity gap. As agents improve, benchmark surfaces that once seemed harmless can become exploitable...
Wei-Jun Luo, Kelvin Luu, Xin-Yi Liu et al.· 0 citations
Reliable Enterprise Agent Deployment (READY), a framework for qualifying AI agents for deployment on enterprise workflows, and provides a basis for comparing agent systems, setting oversight requirements, and making evidence-based deployment decisions.
Veronica Chatrath, Bryan Zhu, Jing-Xuan Fan et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.