An explicit one-dimensional convex bi-objective family is constructed showing that no uniform merit estimate of order $\mathcal O(k^{-(2+\delta)})$ can hold for any fixed $\delta>0$.
Abstract
We investigate a regularized Newton method for unconstrained convex multi-objective optimization with twice continuously differentiable objectives whose Hessians are Lipschitz continuous. At each iteration, the method minimizes the quadratically regularized max-envelope of the local quadratic models. Using a Tanabe-type merit function, we prove that this merit decays at the global asymptotic rate $o(1/k^2)$ under the compactness assumption on the initial component-wise lower level set. This result also covers the single-objective case as a special case. Finally, we construct an explicit one-dimensional convex bi-objective family showing that no uniform merit estimate of order $\mathcal O(k^{-(2+\delta)})$ can hold for any fixed $\delta>0$. Thus the exponent $2$ is essentially sharp in the uniform polynomial sense, despite the $o(1/k^2)$ decay on each fixed trajectory.
A variant of the cubic-regularized Newton method for nonconvex optimization that is parameter-free in that it requires no prior knowledge of problem-dependent parameters is analyzed, and an oracle complexity bound is derived for finding an $(varepsilon, \delta)-second-order stationary point.
A lower bound of $\Omega(\,\frac{d^2}{\log(d+1)})$ on the oracle complexity in this setting is provided, to close this gap dating back to 1996, up to polylogarithmic factors.
This paper shows that AINE can find an $\epsilon$-solution in the inexact second-order oracle (ISO) complexity of $\delta/\epsilon)^{1/2} + (L_2/\epsilon)^{2/7} )$ when the Hessian is $L_2$-Lipschitz continuous, and establishes matching oracle complexity lower bounds for both setups.
Lesi Chen, Chengchang Liu, Luo Luo et al.· 1 citation
We prove an $\Omega(L/T)$ lower bound for the convergence rate of minimization in the class of functions that are convex and $L$-smooth relative to negative entropy on the standard $d$-simplex, valid for every first-order method when $d = \Omega(T^2)$. In particular, this shows that mirror descent is optimal up to a lo...
Jacob M. Aguirre, Dmitrii M. Ostrovskii· arXiv.org· 0 citations
We study the last iterate of the projected subGradient Method (sGM) for convex Lipschitz objectives defined on $\mathbb{R}^d$. We prove that, for a finite horizon $n$ and a constant stepsize $\eta=\Theta(1/\sqrt n)$, the last iterate achieves an optimization error of order $d/\sqrt n$, showing that the extra $\log n$ f...
A global iteration complexity bound is established for the smallest gradient norm among the first $k$ iterates for the smallest gradient norm among the first $k$ iterates when the initial sublevel set is bounded.
Lijun Ding, Jin-Wen Yang, Baoyu Zhou· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.