RuVerBench is introduced, the first benchmark for assessing LaaJ reliability in rubric verification for agentic scenarios, and the impact of key LaaJ strategies, including prompt design, batching, and majority voting, on rubric verification is analyzed.
DramaChain Bench is presented, the first short-drama benchmark that evaluates every stage of the complete production chain, and confirms that final episode quality is not governed by video generation alone.
Haoyuan Shi, Ming-Tao Chen, Shu Jiang et al.· 0 citations
This work introduces RepoProbe, a novel benchmark for evaluating repository-level code understanding through open-ended Q&A using GitHub Discussions, which focuses on open-ended architectural inquiries rather than defect reporting and proposes a Checklist-Based Verification Protocol that decomposes answers into atomic,...
TrajDebug is proposed, an error-lifecycle tracing framework that addresses long-trajectory error discovery with multi-granularity history compression and evidence-based error identification, and supports critical attribution by tracing each error's resolution status and terminal impact.
Yunjia Qi, Zehua Yin, Xin Shi et al.· 4 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.