#machine learning
May 2026
OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling
A dynamic per-layer scalar derived by adapting the LARS/LAMB trust-ratio principle to the orthogonalized setting, where the standard denominator candidates---the raw momentum norm or the polar-factor norm---either live in the wrong unit space or carry no update-scale information.
Yuxuan Lou, Yang You
· arXiv.org · 1 citation