When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay
Together, these results isolate a single governing quantity for scale-invariant optimization, providing a precise and actionable lens on training dynamics, optimizer behavior, and schedule design in modern deep learning.
H. Amin, Wei-Kai Chang, Rajiv Khanna
· 0 citations