This work revisit hallucination in MLLMs from the perspective of reasoning generation dynamics and proposes an entropy-aware self-introspective framework for MLLM reasoning, arguing that entropy is not merely a static measure of uncertainty but a structured signal that reflects how modality information is integrated throughout decoding.
Abstract
Multimodal large language models (MLLMs) have demonstrated remarkable progress on cross-modal reasoning tasks, yet they continue to suffer from hallucinations in long-chain reasoning processes. In this work, we revisit hallucination in MLLMs from the perspective of reasoning generation dynamics. We argue that entropy is not merely a static measure of uncertainty but a structured signal that reflects how modality information is integrated throughout decoding. Through systematic analysis, we identify two characteristic entropy patterns associated with hallucination behaviors: (1) a generally high-confidence regime corresponds to premature convergence to inappropriate multimodal interactions; and (2) local abrupt entropy transitions are associated with reasoning instability and hallucination onset. Building on these observations, we propose an entropy-aware self-introspective framework for MLLM reasoning. Our method continuously tracks entropy dynamics during decoding and performs targeted visual re-alignment when suspicious hallucination states are detected, encouraging the model to re-examine salient visual evidence at critical reasoning stages. Extensive experiments across six multimodal reasoning benchmarks spanning nine distinct evaluation settings demonstrate that our approach delivers robust improvements in reasoning accuracy and hallucination robustness, while providing new insights into the generation dynamics underlying MLLM failures. The code is available at: https://github.com/jden4524/SelfRetrospectiveDecoding.
Large Multimodal Reasoning Models (LMRMs) have recently shown strong capabilities in complex visual understanding and multi-step inference. However, they remain highly vulnerable to multimodal hallucinations over extended reasoning trajectories. In this work, we study this failure as a state-dynamics problem: as reason...
Xiao-Ye Qu, Yan-Zhe Wu, Zhi-Hao Xie et al.· Proceedings of the 34th ACM...· 0 citations
Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in vision–language reasoning. However, the hallucination issue continues to limit their reliability in high-risk scenarios. Existing research predominantly focuses on post-hoc output-layer correction and heuristic interventions, while few...
Jing Xiao, Hao-Sen Wang, Xiao-Wang Zhang et al.· Proceedings of the 34th ACM...· 0 citations
The key innovation of SOPO is a Socratic feedback mechanism, where a teacher model guides the reasoning trajectories during rollout by providing indirect hints inferred from the ground truth, enabling the model to generate more effective reasoning trajectories for hard examples, thus alleviating reward sparsity.
Jia-Wei Dong, Zi-Long Bai, Peng-Tian Zhu et al.· 0 citations
Mental simulation, the ability to reason by constructing and updating internal visual states, is central to flexible human cognition, but remains elusive in current multimodal large language models. Latent multimodal reasoning offers a promising route toward this capability by interleaving text with continuous visual l...
Luo Mi, Rohan Dugad, Zi-Hui Xue et al.· 0 citations
PHRBench is introduced, a controlled benchmark for behaviorally structured PHR across four domains and 18 large language models, and it is found that properties of the hallucinated prompt contain substantial predictive signal for successful recovery, with a lightweight predictor achieving an AUROC of 0.847.
Ling-Hao Meng, Feng He, Xuan Yang et al.· 0 citations
Large language models (LLMs) are widely used for information access, content production, and agent-based task execution, but their outputs remain vulnerable to subtle, high-risk hallucinations. To systematically evaluate and advance post-generation hallucination detection, we introduce HaluCheck, a large-scale, bilingu...
Han-Yu Qiu, Ze-Li Liu, Zhe Zhao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.