GUI agents often encounter dynamic anomalies when deployed on Android devices, from unexpected pop-ups to action misuse, yet existing benchmarks lack systematic evaluation of agent robustness against runtime anomalies. We introduce AnTrap, a comprehensive benchmark that injects dynamic perturbations into agent executio...
GUI-CC is introduced, a benchmark that evaluates contextual consistency of GUI world models as agent environments rather than isolated next-screen predictors and evaluates transition fidelity, transition plausibility, contextual consistency, and task progress.
Linbo Fu, Zheyuan Yang, Tian-Hui Zhang et al.· 0 citations
This work introduces Sci-VBench, a comprehensive benchmark for evaluating knowledge- and reasoning-intensive video generation across scientific domains, and establishes a rubric-based evaluation protocol, showing that advances in visual realism have not yet translated into reliable modeling of scientific and causal dyn...
Diandian Zhang, Tingyu Song, Linbo Fu et al.· 1 citation
We introduce MMSciCode, a comprehensive expert-level, multilingual multi-discipline benchmark for evaluating foundation models in scientific code generation. It includes 624 expert-annotated research coding problems spanning six core scientific disciplines. Compared to prior benchmarks, MMSciCode features three key adv...
Xue Xia, Zheyuan Yang, Arman Cohan et al.· Annual Meeting of the Associ...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
