Polyak--Ruppert averaging gives SGD and momentum methods the same asymptotic covariance. We characterize how fast finite runs reach it and how far they can be parallelized. For strongly convex quadratics with additive noise, we prove an online--offline bridge: the empirical risk minimizer on the samples of a tail avera...
Alexandra Suvroikova, D. Dvinskikh, Y. Dorn et al.· 0 citations
We study zeroth-order optimization of convex and strongly convex functions with H\"older smoothness of order $\beta>2$ from values corrupted by random noise with bias independent of the method's random directions, a Lipschitz systematic error and a bounded adversarial error. We measure the number of function evaluation...
Т. В. Логінов, D. Dvinskikh, Nazarii Tupitsa et al.· 0 citations
We study second-order methods for convex stochastic optimization, where gradients and Hessians are available only through stochastic estimates with variances $\sigma_1^2$ and $\sigma_2^2$, respectively. First, we propose the Stochastic Contracting Cubic Newton method. At each iteration, it minimizes a cubic model with...
A. Agafonov, Аслан Нажмудинович Ашабоков, D. Kamzolov et al.· 0 citations
We study zeroth-order optimization of convex and strongly convex functions with H\"older smoothness of order $\beta>2$ from values corrupted by random noise with bias independent of the method's random directions, a Lipschitz systematic error and a bounded adversarial error. We measure the number of function evaluation...
Т. В. Логінов, D. Dvinskikh, Nazarii Tupitsa et al.· 0 citations
We develop an accelerated stochastic method for convex $(H_0,H_1)$-smooth optimization under heavy-tailed noise. The unbiased oracle has a finite $p$-th noise moment, $1<p\le2$, with constant, gradient-dependent, and gap-dependent terms. Our method combines accelerated updates with clipping, projection, and phase resta...
We study entropy-regularized linear programs with sparse affine constraints and compare two exact dual representations induced by whether a redundant normalization constraint is retained or eliminated. Eliminating it yields a partially separable sum-exp dual with sparse affine factors but unbounded curvature. Our main...
I. Podlipnova, Maxim Mashtaler, A. Agafonov et al.· 0 citations
A two-phase accelerated method that achieves, with high probability, an accelerated optimization contribution and smooth-subclass-optimal statistical dependence on accuracy, up to logarithmic and generalized-smoothness factors.
D. Dvinskikh, A. Gasnikov, A. Lobanov et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
