Liam Storan

Group-Invariant Statistics Determine Embedding Geometry: Harmonic Analysis of Representations from Bach to the Night Sky

The representations that language models learn for concepts such as months, weekdays, and places display consistent geometric structure: circles and saddle-shaped"Pringle"manifolds. Recent work traced these structures to $\textit{translation symmetry}$ in word co-occurrence statistics, deriving the observed Fourier geo...