HappyWorld-Bench is introduced, a comprehensive benchmark that evaluates whether generated worlds remain reliable as agents interact with them, and highlights the need to evaluate world models not only by visual quality, but also by state consistency and the correctness of their responses to actions and interventions.
Zhi-Qi Bai, Ju-Nai Cai, Yi-Xin Chen et al.· 0 citations
VibeMemBench exposes the gap between the useful experience that repository history holds and the experience existing memory systems deliver for repository coding tasks.
Li-Yang Fan, Yingcheng Shi, Yong-Bin Li et al.· 0 citations
RubricRM is proposed, a pairwise generative reward modeling framework that first produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images.
Zi-Jian Kan, Wei Wang, Long Luo et al.· 0 citations
FinIndices is a large-scale benchmark evaluating data-processing fidelity over uncropped financial statements (up to 32K tokens) and yields substantial zero-hint gains, validating that structured logic can be partially restored via data-centric alignment.
Results support treating PHD as a distinct target for evaluating how LLMs formulate investigative directions when final conclusions are withheld, with gains for several lower-performing models on HypoArena but regressions for other systems, including a top-performing model.
Tianyun Zhong, Wangyi Jiang, Wei Wang et al.· arXiv.org· 0 citations
Evaluation of 19 multimodal large language models shows that current MLLMs still cannot reliably make justified and evidence-bounded inferences from experimental results, which is an essential capability in real scientific discovery.
Tao Han, Yucheng Zhang, Jing-Hang Wang et al.· 1 citation
Solution Hacking is identified, a failure mode in which an LLM reaches the correct answer through invalid shortcuts, such as numerical search, enumeration, guessing, or answer-first verification, without providing a valid task-targeted derivation.
Xuanhua Ren, Weiqi Zhai, Tianle Pu et al.· 0 citations
A diagnostic benchmark for open-domain, open-form, long-horizon counterfactual causal reasoning, containing 220 what-if questions across STEM, HSS, and Hybrid scenarios, and finds that WhatIfBench remains far from saturated: even the strongest model reaches only a 64.62% final score.
Yu-Cheng Wang, Yuetian Du, Zheng Liu et al.· 0 citations
Living-Harness is proposed, a self-evolving agent harness that converts each completed trajectory and its evaluator signals into posterior evidence for bounded harness updates, and supports retrieval-only reuse of the evolved harness state across model backbones.
Yuetian Du, Yucheng Wang, Helsing Xu et al.· arXiv.org· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.