Skip to content

Author

Martin T. Wells

1 paper indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization

Borders reveal a noise--drift tradeoff: in noise-dominated regimes, first-moment averaging and adaptive preconditioning can yield favorable upper guarantees, whereas in drift-dominated regimes, stale first-moment information and preconditioning can enlarge Adam's tracking guarantee, potentially allowing vanilla SGD to...

Sharan Sahu, Abir Sarkar, Cameron J. Hogan et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.