Skip to content
Book Open access

Ensuring Semantic Convergence to Visual Evidence for Hallucination Mitigation in MLLMs

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 44 references

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in vision–language reasoning. However, the hallucination issue continues to limit their reliability in high-risk scenarios. Existing research predominantly focuses on post-hoc output-layer correction and heuristic interventions, while few studies investigate the underlying hallucination mechanisms from the perspective of internal semantic evolution. Through empirical analysis, this paper reveals that the reasoning process of mainstream MLLMs follows a three-phase semantic evolution pattern—explorati-on, anchoring, and convergence. Hallucinations arise when the semantic evolution fails to converge toward the visual evidence. We theoretically prove that ensuring such convergence effectively suppresses hallucination generation and propose a phase-specific intervention method: during the anchoring phase,visual information is strengthened to guide the model toward reasoning directions grounded in true visual evidence. In the convergence stage, critical visual details are enhanced to ensure convergence to the correct visual evidence.The proposed method requires no additional training, can be flexibly adapted to various architectures, and extensive experiments validate its effectiveness in mitigating hallucinations.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.