Vision-language models (VLMs) have demonstrated excellent performance in visual tasks, but their visual causal reasoning capabilities still lack reliable evaluation. Existing evaluations struggle to distinguish whether a model is performing causal reasoning based on visual evidence or relying on statistical correlation...
Direct-decision models turn text into low-latency structured labels and scores, making them attractive for classification and automatic evaluation. Yet reliability requires more than accuracy: a model must also use the ordinal decision scale supplied by the user faithfully. We analyze JEV~1.13 and three open KEV models...
Tian-Xiang Gao, Jin-Zhe Li, Zhiyuan Li et al.· 0 citations
These results show that reliable scene-text recognition requires VLMs to balance visual character evidence with contextual information, preserving clear text while using context mainly when the visual evidence is uncertain.
It is found that competence dominates human-directed evaluations, while many \textit{other} attributions describe humans as epistemic, cultural, or embodied subjects, and suggest that bias in agent societies should be studied not only as isolated model output, but also as a discourse process.
Multi-turn interaction creates a feedback process in which an LLM's previous responses become context for later behavior. Prior work shows substantial multi-turn degradation and that assistant-generated history can affect later behavior. However, it remains unclear how these effects manifest across models, tasks, turns...
Jinnan Li, Zheren Fu, Yue Wang et al.· 0 citations
VeriOCRBench, a 1,800-sample human-verified benchmark built from source images drawn from 8 OCR-related datasets and spanning 8 real-world image domains, with controlled, image-grounded diagnostic tasks, is introduced, exposing a critical reliability gap in current OCR reasoning systems.
RPCBench, a benchmark for evaluating Recommender-Premise Critique: the ability to detect, diagnose, and properly handle faulty premises in natural-language recommendation requests, is introduced and a fine-grained evaluation framework that measures proactive detection, error localization, post-detection handling strate...
This work proposes MMPCBench, a comprehensive framework for evaluating MLLMs' proactive critique competence, which adopts a hierarchical evaluation protocol to measure models' error detection, diagnosis and resolution performance, and applies alignment-aware metrics to assess the coherence between internal reasoning an...
Jin-Zhe Li, Geng-Xu Li, Jinnan Li et al.· 0 citations
Experiments with representative closed-source and open-source MLLMs show that OCR-grounded meta-reasoning remains far from saturated: models struggle with visible-rule application and layout-sensitive inference, while process-compliant rationales can accompany incorrect final answers under exact-match evaluation.
By eliminating routers entirely, CD-LoRA employs a consistency-driven alignment mechanism to enforce representation congruence across tasks in a shared low-rank space, which fosters robust, task-agnostic features without explicit partitioning overhead.
CaSKG, a counterfactual-causal skill graph framework that calibrates procedural relations before retrieval, is proposed, position edge-confidence calibration as an effective route to compact and executable skill retrieval at scale.
Zhiyuan Li, Lin-Yuan Gao, Xue-Chun Ding et al.· 1 citation
This work applies Top-K Sparse Autoencoders to the intermediate representations of DeepSeek-R1-Distill-Qwen-7B and examines the model's divergent behaviors across math-solving tasks of three distinct difficulty levels, identifying a clear distinction in how the model functions under two reasoning modes.
Bo Cheng, Qiaolin Lu, Yi Chang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
