Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders
The results suggest that architectural routing mechanisms may have negligible impact on core semantic understanding, with representational divergence confined to extreme structural margins.