We establish matching lower bounds for deterministic first-order optimization of globally Lipschitz geodesically convex functions on hyperbolic space. A global minimizer lies within distance $r$ of a known point, each query returns an exact function value and a selected Riemannian subgradient, and the target objective...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 1 citation
We consider stochastic nonconvex optimization with sample-wise Lipschitz gradients and Hessians, and reusable opaque sample handles. Gradient and complete-Hessian evaluations are charged separately, including both endpoints of a same-sample difference. For fixed positive problem parameters and in the high-dimensional w...
Qian Chen, Wen-Dao Wu, Hai-Han Zhang et al.· 0 citations
The randomized component-oracle complexity of finite-sum nonconvex-concave optimization with a bounded dual domain is characterized, taking the worst case over admissible known dual domains and finite dimensions.
Wen-Dao Wu, Hai-Han Zhang, Qian Chen et al.· 0 citations
We characterize the oracle complexity of finding first-order stationary points in smooth nonconvex--strongly-concave (NC--SC) minimax optimization when each query provides the full Hessian in addition to the function value and gradient. For $\Phi(x)=\max_y f(x,y)$, assume that $f$ has an $L$-Lipschitz gradient, a $\rho...
Fang-Qing Zhang, Hai-Han Zhang, Wen-Dao Wu et al.· 0 citations
Does randomization improve the worst-case first-order complexity of nonconvex optimization when the objective is very smooth? We prove that, for every fixed finite integer $p\ge3$, finding an $\eps$-stationary point requires $\Omega_p(\eps^{-5/3})$ exact value-and-gradient queries, even for $C^\infty$ objectives with i...
Hai-Han Zhang, Wen-Dao Wu, Fang-Qing Zhang et al.· 0 citations
Tracking changing comparators is a central problem in online learning. For exp-concave losses, previous work established the $T^{1/3}P_T^{2/3}$ dependence on the horizon $T$ and Euclidean path length $P_T$ for fixed dimension, while the optimal cost of dimension remained unresolved. We resolve this question by establis...
Yan-Yi Li, Wen-Dao Wu, Hai-Han Zhang et al.· 0 citations
We study the sample complexity of finding a point with expected hypergradient norm at most $\epsilon>0$ in nonconvex--strongly-convex stochastic bilevel optimization using fresh, globally unbiased first-order samples with bounded variance. For lower-variable smoothness order $p\ge1$, F$^2$SA-$p$ achieves $\widetilde O(...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
We study deterministic adaptive optimization of globally $\beta$-smooth, $\mu$-strongly convex functions using exact scalar function values. Queries and outputs lie in $B_2^d(R)$, and the minimizer lies in $B_2^d(R/2)$. Set $\kappa=\beta/\mu$, $Q=\beta R^2/\epsilon$, and $D_d=(d/\log(ed))^{1/3}$. For sufficiently large...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
We establish a randomized first-order lower bound for finding an $\epsilon$-stationary point, $\|\nabla f(x)\|\le \epsilon$, of a nonconvex function with initial gap at most $\Delta$, $L_1$-Lipschitz gradient, and $L_2$-Lipschitz Hessian. Each oracle call returns the exact function value and gradient. Let $Q_{\mathrm{r...
Hai-Han Zhang, Wen-Dao Wu, Chen-Heng Zhang et al.· 1 citation
We prove a joint stochastic zeroth-order lower bound for Goldstein stationarity on a Euclidean query ball, even when the ball is guaranteed to contain a stationary point. In dimension $d$, let $f=\mathbb{E}[F(\cdot;\xi)]$, assume $\mathbb{E}[\operatorname{Lip}(F(\cdot;\xi))^2]\le L_0^2$, and bound the initial objective...
We establish tight randomized higher-order oracle complexity for finding first-order stationary points of nonconvex finite sums. Let $n$ be the number of components, $\Delta>0$ the initial objective-gap bound, $L_p>0$ an individual $p$-th derivative Lipschitz bound, and $\epsilon>0$ the target gradient norm. For every...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 1 citation
We characterize the fresh-gradient oracle complexity of smooth nonconvex-strongly-concave minimax optimization, with matching upper and lower bounds up to logarithmic factors. Let $\Phi(x)=\max_y f(x,y)$, where $f$ is jointly $L$-smooth and $\mu$-strongly concave in $y$ on unconstrained Euclidean domains, and set $\kap...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
