AMO: Operator-level Adaptive Muon Orthogonalization
Operator-level Adaptive Muon Orthogonalization (AMO), an observe-then-commit method that measures weight geometry by operator type early in training and then uses these signals to allocate the NS budget for the remainder of training, delivers consistent improvements over uniform-schedule Muon.
Xin-Lin Zhuang, Panyi Ouyang, Yicheng Li et al.
· 0 citations