Entropy-Adaptive Routing Selection for Multimodal Chain-of-Thought Reasoning

Abstract

Multimodal Chain-of-Thought (MCoT) endeavors to reason across diverse modalities sequentially, facing primary challenges of logical inconsistency with visual evidence and exorbitant computational overhead in long-chain inference. The prompt-based paradigm has been predominantly adopted, relying on zero-shot instructions and task orchestration to generate intermediate rationales. However, semantic inconsistencies between visual and textual contents are commonly encountered, which readily lead to flawed reasoning chains and unreliable inference results. In contrast, learning-based methods align cross-modal information to mitigate reasoning discrepancies, but suffer from substantial training overhead and insufficient labeled datasets. In this study, we introduce EARS-MCoT, a training-free framework centered on Entropy-Adaptive Routing Selection, aiming to bridge the semantic gap between visual and textual modalities while mitigating the computational burden of exhaustive multi-step reasoning. Initially, large language models generate the predictive probability of each token for tokenlevel entropy calculation. Token-level entropy is aggregated via a sliding window mechanism to generate window-level entropy for capturing holistic reasoning fluctuations. Finally, by integrating the Tree-of-Thoughts (ToT) paradigm, EARS-MCoT dynamically optimizes reasoning paths characterized by high entropy and uncertainty. Experimental results demonstrate that EARS-MCoT surpasses strong tree-free baselines on two mainstream visual question answering datasets, achieving superior robustness and efficiency in complex multimodal reasoning tasks.