Spectral Allocation: Why Muon Outperforms Adam, and How to Improve Muon
Spectral-Aware Muon is introduced, which holds the head at the Muon scale and amplifies the bulk using a static spectral prior, and both variants outperform tuned AdamW and Muon (Scion implementation) baselines in all evaluated model-scale and batch-size configurations.
Xiaodong Wu, Wenyi Yu, Chao Zhang et al.
· 0 citations