A Quadratic Lens on Muon's Polar Update: Benefits, Pitfalls, and the Role of Newton-Schulz
PolarGD, the orthogonalized update at the core of Muon, is studied on matrix quadratics, where optimizers can be analyzed in closed form, and update directions under line search are compared to remove the effect of learning-rate tuning.