JudgeProfile is introduced, a framework that dissects LLM evaluation into perception (how a judge compares two responses across specific attributes like clarity, correctness, and detail) and prioritization (how much each attribute influences the final choice) and finds a hidden consensus in perception.
Qiong-Qiong Cao, Kang-Ni Liu, Xuan Kan et al.· 0 citations
LLM judges are inherently subjective, often favoring different responses in pairwise comparison when neither option is objectively wrong. To study this subjectivity, we introduce JudgeProfile, a framework that dissects LLM evaluation into perception (how a judge compares two responses across specific attributes like cl...
Qiong-Qiong Cao, Kang-Ni Liu, Xuan Kan et al.· 0 citations
AIBuildAI-2.5 is introduced, an agentic system that carries out the tree search with LLM agents and proposes a novel LLM-guided tree search, in which a judge scores each candidate on its expected improvement, grounding, and feasibility, and a selector ranks the pool of candidates from these scores and the state of the...
Pei-Jia Qin, Rui-Yi Zhang, Qi Cao et al.· 0 citations
Retrieval assembles repository context by ranking passages for relevance to the current query. A coding agent halfway through an issue has already read much of what such a ranker returns. Relevance is scored per passage, but sufficiency belongs to the set: independently scored passages can fill the budget with support...
Zhe-Xi Feng, Rui-Yi Zhang, Yong-Bo Yang et al.· 0 citations
This work introduces SCALE-QA, a constraint-grounded task QA benchmark for flat unsegmented threads targeting episode integrity failure, and proposes Temporal-Semantic Interleaved Memory Reconstruction (TSIM), which segments the turn stream into coherent episodes and indexes them through a hierarchical multi-view memor...
Zhe-Xi Feng, Rui-Yi Zhang, Yong-Bo Yang et al.· 0 citations
Visuomotor imitation policies can achieve high performance under in-distribution visual conditions yet fail when visually similar objects or receptacles are introduced. We study this behavior as a problem of conditional visual grounding: the visual target required for successful control changes with the manipulation ph...
Vivek Chavan, Peng-Tao Xie, Ya-Huan Shi et al.· 0 citations
Over six months with four state-of-the-art LLM agents configured with web search, aggregate nowcast accuracy is broadly comparable to the institutional and professional benchmarks, with performance varying widely across individual indicators.
Xin-Yue Zhao, Rui-Yi Zhang, Li-Qin Ye et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
