Invariant Atoms: Sparse Coordinates of Local Semantic Geometry in Language Model Representations
This work learns a shared semantic frame and sparse coordinates that reconstruct semantic displacements while suppressing nuisance variation, with anchor-dependent diagonal modulation adjusting atom strengths without sample-specific rotations to support reusable invariant directions as a sparse coordinate system for lo...
Muhammad Ahtesham, Xin Zhong
· 0 citations