High-Probability Guarantees for SGD under $\beta$-Heavy-Tailed Gradient Noise
This work model SGD gradient noise by adopting a Young function that preserves the finiteness of all polynomial moments while allowing tails heavier than sub-Weibull, including lognormal distributions, and establishes concentration inequalities for $\beta$-heavy-tailed noise.
Qi-Jun Tong, Masahiro Ikeda, Ryota Kawasumi
· 0 citations