When Should LLMs Reason? Adaptive Inference Routing via Entropy Dynamics
TL;DR
The proposed EDRM (Entropy Dynamics-based Reasoning Manifold), a lightweight routing framework whose default heuristic requires no router training, supports selective reasoning based on early decoding signals and demonstrates the potential of entropy-based control for efficient LLM inference.
Abstract
Chain-of-thought (CoT) reasoning can improve LLM performance, but its benefits vary across tasks, models, and individual queries. On some tasks, CoT provides marginal or negative accuracy gains while substantially increasing token consumption. We investigate whether early decoding dynamics can help identify when explicit reasoning is useful. Our analysis shows that tasks with larger CoT gains tend to exhibit decreasing early-stage entropy, while low-gain tasks more often show oscillatory or increasing trajectories. These patterns motivate an empirical regime interpretation of decoding behavior. We propose \textbf{EDRM} (Entropy Dynamics-based Reasoning Manifold), a lightweight routing framework whose default heuristic requires no router training. EDRM summarizes early entropy trajectories with a compact, interpretable representation and selects among Direct, Standard, and CoT decoding. Across 15 benchmarks and 7 LLMs, its global and instance-level variants offer \textbf{16--55\%} token savings at comparable accuracy. The Full instance-level configuration combines routing with a short Direct fallback branch; our heuristic-router ablation reports up to \textbf{4.9 percentage points} of additional accuracy from oracle answer selection over routing without fallback. These results support selective reasoning based on early decoding signals and demonstrate the potential of entropy-based control for efficient LLM inference.
