Multimodal large language models (MLLMs) are rapidly evolving toward continuous audio--visual reasoning, creating an urgent need for evaluations that expose their capability limits. Audio--visual captioning is an ideal diagnostic task, yet current benchmarks face a coupled trade-off: whole-caption scores provide covera...
Zhongyu Yang, Jia-Le Tao, Rui-Tao Chen et al.· 0 citations
MechReason is a benchmark derived from real mechanical engineering papers, comprising 12k question-answer pairs with explicit reasoning-chain annotations and 21k visual materials spanning nine evidence types, including statistical charts, parameter tables, engineering drawings, microscopic images, simulation images, sy...
Tengyue Wang, Kang An, Chenxu Du et al.· 0 citations
benchmark provides a focused testbed for measuring whether multimodal models can transform technical visual evidence into sustained, physically valid symbolic reasoning, and provides a focused testbed for measuring whether multimodal models can transform technical visual evidence into sustained, physically valid symbol...
Xinqi Yang, Kang An, Tengyue Wang et al.· 0 citations
MMArch is introduced, a benchmark for architecture and civil engineering spanning ten subdomains and built entirely from figures in peer-reviewed papers, and error analysis shows that failures concentrate in applying principles and combining evidence across figures rather than in locating it, pointing to substantial he...
Chenxu Du, Kang An, Tengyue Wang et al.· 1 citation
Evaluation of representative proprietary and open-source vision--language models reveals substantial performance differences and persistent weaknesses in comparative, technical, and multi-evidence reasoning, demonstrating that strong general visual understanding does not yet guarantee reliable industrial-safety reasoni...
Yuanchi Zhu, Kang An, Tengyue Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
