Jia-Ning Qi

GRRR: The Geometry of Reshaping, Rotation, and Routing in Decoder LLM post-training

It is found that removing the diagonal component usually preserves most of the gains from post-training, suggesting that post-training gains are carried primarily by reconfiguring and extending pretrained pathways rather than by substantially changing singular values of pre-trained models.