Why $\beta_1 = \beta_2$ Is Dynamically Special in Adam
Overall, the results identify memory-scale mismatch as a concrete source of magnitude sensitivity in Adam and provide a mechanistic account of why tied momentum is dynamically distinctive.
Alberto Fernández-Hernández, Cristian Pérez-Corral, José I. Mestre et al.
· 1 citation