The First Few Tokens Matter: Understanding Chain-of-Thought Controllability
This work investigates the source of an entropy collapse in the first few tokens following the opening thinking tag and exploits this mechanism to recover CoT controllability through two inference-time interventions on initial tokens, finding that the relationship with monitorability depends on the evaluation.