Beyond Dense States: Sparse Transcoders as Causally Testable Operators for LLM Latent Reasoning
LSTR (Latent Sparse Transcoder Reasoning), a framework that turns sparse transcoders from post-hoc diagnostic tools into in-loop, intervenable transition components for latent reasoning, and suggests that sparse latent transitions can preserve the compression benefits of latent reasoning while making the resulting trajectories more inspectable and intervenable.
Yadong Wang, Hao-Dong Chen, Yu Tian et al.
· 0 citations