The Hidden Ratio in Adam: Stable Structure, Compression, and Sign Dynamics
The results suggest that tied-$\beta$ Adam admits a simple and approximately stable ratio structure underlying its adaptive behavior and demonstrate its utility for both analysis and efficient implementation.
Yi-He Zhou, Tongtian Zhu, Ying Huo et al.
· 0 citations