RuVerBench is introduced, the first benchmark for assessing LaaJ reliability in rubric verification for agentic scenarios, and the impact of key LaaJ strategies, including prompt design, batching, and majority voting, on rubric verification is analyzed.
This work introduces RepoProbe, a novel benchmark for evaluating repository-level code understanding through open-ended Q&A using GitHub Discussions, which focuses on open-ended architectural inquiries rather than defect reporting and proposes a Checklist-Based Verification Protocol that decomposes answers into atomic,...
Juris Policy Optimization (JPO), a post-training framework for structured legal reasoning in Chinese criminal judgment prediction, is proposed and experiments show that JPO consistently improves both judgment prediction and reasoning quality over supervised fine-tuning and reinforcement learning baselines.
Zhao-Lu Kang, Yan-Tao Liu, Tai-Long Luo et al.· 1 citation
SCEval (Structure-Corruption Evaluation) a diagnostic evaluation protocol that keeps the question, answer space, and modality channels fixed while applying controlled structural corruptions to text, vision, and audio individually and jointly is introduced.
Zhaolu Kang, Mei-Xin Wu, Yu Xue et al.· 1 citation
TrajDebug is proposed, an error-lifecycle tracing framework that addresses long-trajectory error discovery with multi-granularity history compression and evidence-based error identification, and supports critical attribution by tracing each error's resolution status and terminal impact.
Yunjia Qi, Zehua Yin, Xin Shi et al.· 4 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.