Jul 2026
The Convergence Behavior of Adam under Heavy-Tailed Noise
A discounted regret analysis for Adam is developed, without restrictive parameter coupling, and the first convergence guarantees for the plain vector-form Adam optimizer under heavy-tailed stochastic noise are established.
Yijiang Pang
· arXiv.org · 1 citation