Skip to content

The Model Knows When to Stop: Training-Free Early Stopping for Long-Context Reading

Sep 2026 · arXiv.org · Vol abs/2609.34590 · 0 citations · 21 references
Computer Science

TL;DR

Answer-Convergence Stopping (ACS) is introduced, a training-free stopping rule that measures rather than asks, and reveals that by properly utilizing the output signals of frozen models, it can achieve favorable behaviors like adaptive stopping without the need for additional training.

Abstract

Language models often process long inputs sequentially in chunks, but continuing to read after sufficient evidence has been acquired wastes computation. Existing stopping mechanisms either learn sufficiency from internal activations or train an exit gate, while a simpler alternative asks the model whether it has read enough. We introduce Answer-Convergence Stopping (ACS), a training-free stopping rule that measures rather than asks. After each chunk, it probes the frozen model's current answer state and stops when that state is both confident and stable. The rule requires only output-side generation and token log probabilities, has no trained components, and uses one shared configuration across models and benchmarks. Because a stopping policy can save computation simply by stopping too early, we evaluate the stopping decision itself using evidence position where available. On the full LongBench-v2 with two frontier models, ACS is the only stopping policy that matches or exceeds full-reading accuracy. Furthermore, across 250 S-NIAH questions, the premature stopping rate for ACS across five models from two families ranges from 0% to 12%, compared to 8.4% to 45.6% for the verbalized gate. Taken together, ACS reveals that by properly utilizing the output signals of frozen models, we can achieve favorable behaviors like adaptive stopping without the need for additional training.

View source

Similar papers

#machine learning Preprint Sep 2026

In-Context Binding Capacity in Language Models

This work derives why interference can lower measured capacity by reducing single-binding recall even when the load-dependent recall profile is unchanged, and characterize capacity at the model's query interface.

Manas Venkata Sai Ravulapalli, Samrath Singh Chadha · 0 citations
#machine learning Preprint Sep 2026

RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory

RecurTrace introduces Loop Memory Attention, which lets each looped layer attend to its own states from previous iterations along the loop-time axis, so the model can revisit earlier computations instead of relying on the latest state alone.

Yu-Xiang Wang, Kun-Yu Feng, Ying-Da Shen et al. · 7 citations

Scoring the Wrong Question: Readout Failures in Constrained-Option Evaluation

Constrained-option scoring reads a model's probabilities for a fixed set of permitted answers, so it returns a score even when the model is about to write something else. We study a prompt that quotes a multiple-choice item, ending in the item's own answer instruction, and asks for a forecast of whether a reader model...

Jia-Xuan Guo, Ke-Jia Zhang, Shuo Xin et al. · 1 citation

Related blog posts

MIT News · Artificial Intelligence Aug 27, 2026

Looking beyond natural sequences

A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.