Polyak--Ruppert averaging gives SGD and momentum methods the same asymptotic covariance. We characterize how fast finite runs reach it and how far they can be parallelized. For strongly convex quadratics with additive noise, we prove an online--offline bridge: the empirical risk minimizer on the samples of a tail avera...
Alexandra Suvroikova, D. Dvinskikh, Y. Dorn et al.· 0 citations
We study zeroth-order optimization of convex and strongly convex functions with H\"older smoothness of order $\beta>2$ from values corrupted by random noise with bias independent of the method's random directions, a Lipschitz systematic error and a bounded adversarial error. We measure the number of function evaluation...
Т. В. Логінов, D. Dvinskikh, Nazarii Tupitsa et al.· 0 citations
We study zeroth-order optimization of convex and strongly convex functions with H\"older smoothness of order $\beta>2$ from values corrupted by random noise with bias independent of the method's random directions, a Lipschitz systematic error and a bounded adversarial error. We measure the number of function evaluation...
Т. В. Логінов, D. Dvinskikh, Nazarii Tupitsa et al.· 0 citations
We develop accelerated gradient-free methods for stochastic convex optimization with constraints defined by expectations. Our batched primal-dual sliding method uses two-point evaluations sharing a random sample and guarantees expected objective error and expected maximum constraint violation at most $\varepsilon$. It...
Vadim Abronin, A. Gasnikov, D. Dvinskikh· 0 citations
We develop an accelerated stochastic method for convex $(H_0,H_1)$-smooth optimization under heavy-tailed noise. The unbiased oracle has a finite $p$-th noise moment, $1<p\le2$, with constant, gradient-dependent, and gap-dependent terms. Our method combines accelerated updates with clipping, projection, and phase resta...
We establish the optimal query complexity of smooth convex quadratic optimization from exact function values. For dimension $d$, smoothness $L$, and minimizer radius $R$, the sharp rate at small relative error is $\Theta(d\min\{d,\sqrt{LR^2/\varepsilon}\})$. The lower bound has no logarithmic loss and holds even for ad...
Vadim Abronin, A. Gasnikov, D. Dvinskikh· 0 citations
We study the query complexity of optimization with exact scalar-value information. For globally $L$-smooth convex functions on $\mathbb R^d$ with a minimizer in a Euclidean ball of radius $R$, we prove the lower bound $\Omega(d\min\{d,\sqrt{LR^2/\varepsilon}\})$ for adaptive randomized algorithms in the stated accuracy...
Y. Dorn, D. Dvinskikh, Т. В. Логінов et al.· 0 citations
It is shown that for this mismatched problem the minimax value of the gap is of order $LR^2/N^3$ up to logarithmic factors, for deterministic and for randomized methods alike, and already in dimension proportional to the number of queries.
Т. В. Логінов, A. Gasnikov, Yuriy Dorn et al.· 0 citations
This work derives a finite-sample comparison of clipped and unclipped Polyak-Ruppert averaged SGD under finite conditional $p$-th moments, and gives explicit accuracy and confidence conditions under which, for $p>2$, the Gaussian term dominates the unclipped deviation bound, so clipping need not improve its leading ord...
A. Suvorikova, E. Gladin, D. Dvinskikh et al.· 0 citations
A two-phase accelerated method that achieves, with high probability, an accelerated optimization contribution and smooth-subclass-optimal statistical dependence on accuracy, up to logarithmic and generalized-smoothness factors.
D. Dvinskikh, A. Gasnikov, A. Lobanov et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
