Ceiling of a Task: When Can a Transformer Succeed Without Its Chain of Thought?
Reasoning models generate long chains of thought before they answer, yet it is debated whether the content of these chains does real computational work or is largely decorative. We study this question by viewing a transformer as a shallow circuit. One forward pass through a fixed number of layers has constant depth, so...
Jia-Shu He, Jin-Xuan Fan, Xiao Xiao et al.
· 0 citations