When Models Don't Manipulate Manifolds: The Geometry of a Comparison Task
This work characterize precisely the geometry of computation in a number-comparison task, as an abstraction of comparison for decision making, and how models utilize geometry in an elegant fashion to implement it, demonstrating that the manifold hypothesis can co-exist with linear representations.