Skip to content
Preprint

Entropogram-guided Visual Realignment Decoding: Capturing the Rhythm of Multimodal Reasoning

Nov 2026 · 0 citations · 43 references
Computer Science

TL;DR

This work revisit hallucination in MLLMs from the perspective of reasoning generation dynamics and proposes an entropy-aware self-introspective framework for MLLM reasoning, arguing that entropy is not merely a static measure of uncertainty but a structured signal that reflects how modality information is integrated throughout decoding.

Abstract

Multimodal large language models (MLLMs) have demonstrated remarkable progress on cross-modal reasoning tasks, yet they continue to suffer from hallucinations in long-chain reasoning processes. In this work, we revisit hallucination in MLLMs from the perspective of reasoning generation dynamics. We argue that entropy is not merely a static measure of uncertainty but a structured signal that reflects how modality information is integrated throughout decoding. Through systematic analysis, we identify two characteristic entropy patterns associated with hallucination behaviors: (1) a generally high-confidence regime corresponds to premature convergence to inappropriate multimodal interactions; and (2) local abrupt entropy transitions are associated with reasoning instability and hallucination onset. Building on these observations, we propose an entropy-aware self-introspective framework for MLLM reasoning. Our method continuously tracks entropy dynamics during decoding and performs targeted visual re-alignment when suspicious hallucination states are detected, encouraging the model to re-examine salient visual evidence at critical reasoning stages. Extensive experiments across six multimodal reasoning benchmarks spanning nine distinct evaluation settings demonstrate that our approach delivers robust improvements in reasoning accuracy and hallucination robustness, while providing new insights into the generation dynamics underlying MLLM failures. The code is available at: https://github.com/jden4524/SelfRetrospectiveDecoding.

View source

Similar papers

Book Open access Nov 2026

Mitigating Hallucinations in Large Multimodal Reasoning Models by Regulating Parametric Prior Drift

Large Multimodal Reasoning Models (LMRMs) have recently shown strong capabilities in complex visual understanding and multi-step inference. However, they remain highly vulnerable to multimodal hallucinations over extended reasoning trajectories. In this work, we study this failure as a state-dynamics problem: as reason...

Xiao-Ye Qu, Yan-Zhe Wu, Zhi-Hao Xie et al. · 0 citations
Book Open access Nov 2026

Ensuring Semantic Convergence to Visual Evidence for Hallucination Mitigation in MLLMs

Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in vision–language reasoning. However, the hallucination issue continues to limit their reliability in high-risk scenarios. Existing research predominantly focuses on post-hoc output-layer correction and heuristic interventions, while few...

Jing Xiao, Hao-Sen Wang, Xiao-Wang Zhang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

SOPO: Socratic Guided Policy Optimization for Span-Level Hallucination Detection

The key innovation of SOPO is a Socratic feedback mechanism, where a teacher model guides the reasoning trajectories during rollout by providing indirect hints inferred from the ground truth, enabling the model to generate more effective reasoning trajectories for hard examples, thus alleviating reward sparsity.

Jia-Wei Dong, Zi-Long Bai, Peng-Tian Zhu et al. · 0 citations
#artificial intelligence Preprint Nov 2026

Mental Imagery That Matters: Curing Latent Collapse in Multimodal Reasoning

Mental simulation, the ability to reason by constructing and updating internal visual states, is central to flexible human cognition, but remains elusive in current multimodal large language models. Latent multimodal reasoning offers a promising route toward this capability by interleaving text with continuous visual l...

Luo Mi, Rohan Dugad, Zi-Hui Xue et al. · 0 citations
#artificial intelligence Preprint Oct 2026

PHRBench: A Behavioral Evaluation of Post-Hallucination Reasoning in LLMs

PHRBench is introduced, a controlled benchmark for behaviorally structured PHR across four domains and 18 large language models, and it is found that properties of the hallucinated prompt contain substantial predictive signal for successful recovery, with a lightweight predictor achieving an AUROC of 0.847.

Ling-Hao Meng, Feng He, Xuan Yang et al. · 0 citations
Preprint Nov 2026

From Twisted Facts to Rogue Agents: Can LLMs Reliably Detect Multi-Dimensional Hallucinations?

Large language models (LLMs) are widely used for information access, content production, and agent-based task execution, but their outputs remain vulnerable to subtle, high-risk hallucinations. To systematically evaluate and advance post-generation hallucination detection, we introduce HaluCheck, a large-scale, bilingu...

Han-Yu Qiu, Ze-Li Liu, Zhe Zhao et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.