Mar 2026· arXiv.org· Vol abs/2603.26737· 1 citation· 48 references
Computer Science
TL;DR
A question-relevant saliency map identifies and organizes key visual regions, explicitly modeling the spatial distribution of visual importance, and induces a curriculum-like semantic progression from primary to secondary cues, validating structured and sequential visual cognition.
Abstract
Current multimodal LLMs encode images as static visual prefixes and rely on text-based reasoning, lacking goal-driven and adaptive visual access. Inspired by human visual perception-where attention is selectively and sequentially shifted from the most informative regions to secondary cues-we propose Structural Sequential Visual CoT SSV-CoT. First, a question-relevant saliency map identifies and organizes key visual regions, explicitly modeling the spatial distribution of visual importance. Second, reasoning is performed following this discriminative order, inducing a curriculum-like semantic progression from primary to secondary cues. This method is trained end-to-end, using text cot and answer supervision, without relying on region-level annotations or specialized external tools. Experiments on diverse visual reasoning benchmarks show gains, validating structured and sequential visual cognition.
Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments. By generating intermediate reasoning images, Visual CoT provides an intuitive mechanism for visual foresight but introduces substantial inference overhead, which is partic...
Xiao-Yu Zhu, Xin-Ke Deng, Suresh Taddewadikar et al.· 0 citations
LIRSeg is proposed, which fully replaces explicit CoT with a compact set of learnable latent tokens for reasoning segmentation, and introduces three complementary mechanisms from an information perspective: extreme-advantage sampling for selecting informative training signals, decoupled exploration-stability updates fo...
Tian-Hang Guo, Yu-Lin He, Wei Chen et al.· 0 citations
Across six diverse visual reasoning tasks including multi-view spatial reasoning and collision prediction, ReImaGin consistently outperforms both text-only reasoning and specialist vision-tool baselines, with gains of up to 25\%, demonstrating the advantage of flexible, generative visual reasoning.
Nishad Singhi, H. P. Rodriguez, Aditya Arora et al.· 0 citations
Think with Structured Grounding (TwSG), a novel fine-grained image perception framework designed to internalize complex images's tool-use capabilities within the model, is proposed, endowing models with native fine-grained region description and flexible reasoning capabilities.
Chang-Jiang Jiang, Qian-Nian Zhao, Lei Xin et al.· 2 citations
This work designs a semantic structure-based feature refinement module to adaptively modulate subject-centric and contextual visual features and incorporates a visual semantic consistency verification module, which leverages a subject-aware contrastive learning strategy to constrain and verify the semantic corresponden...
Jie-Peng Cai, Zhen Xu, Tie-Song Zhao et al.· Proceedings of the Thirty-Fi...· 0 citations
This work investigates cross-modal information flow in a video-based generative multiple-choice-like setting by applying a layer-wise causal intervention on video-text attention pathways and identifies a distinct pattern in temporal reasoning.
Davide Testa, Hugh Mee Wong, Alessandro Lenci et al.· 0 citations
Known for his clear and elegant writing style, Bertsekas shaped fields from control and optimization to large-scale computation and artificial intelligence.
