Hugh Van Deventer

The First Few Tokens Matter: Understanding Chain-of-Thought Controllability

This work investigates the source of an entropy collapse in the first few tokens following the opening thinking tag and exploits this mechanism to recover CoT controllability through two inference-time interventions on initial tokens, finding that the relationship with monitorability depends on the evaluation.