The Hidden Ratio in Adam: Stable Structure, Compression, and Sign Dynamics
The results suggest that tied-$\beta$ Adam admits a simple and approximately stable ratio structure underlying its adaptive behavior and demonstrate its utility for both analysis and efficient implementation.