We prove randomized matrix-vector query lower bounds for two normalized matrix-game geometries: a Euclidean unit ball against a probability simplex, with row norms at most one, and two probability simplices, with entries of absolute value at most one. Each query returns $(Ax,A^\top y)$ for arbitrary real vectors. The a...
The optimal accuracy dependence for smooth nonconvex--strongly-convex bilevel optimization with stochastic first-order oracles is characterized and an $\Omega(\epsilon^{-6})$ lower bound matching the accuracy exponent of existing upper bounds is proved.
Hai-Han Zhang, Wen-Dao Wu, Chen-Heng Zhang et al.· 0 citations
We establish matching lower bounds for deterministic first-order optimization of globally Lipschitz geodesically convex functions on hyperbolic space. A global minimizer lies within distance $r$ of a known point, each query returns an exact function value and a selected Riemannian subgradient, and the target objective...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 1 citation
The new first-order lower bound is specifically robust to persistent sample reuse and to complete-Hessian access, and uses a slowly parameterized conservative sign chain, a third-order reverse stick-breaking localization inequality, and stateful relative-entropy accounting for reusable handles.
Qian Chen, Wen-Dao Wu, Hai-Han Zhang et al.· 1 citation
The randomized component-oracle complexity of finite-sum nonconvex-concave optimization with a bounded dual domain is characterized, taking the worst case over admissible known dual domains and finite dimensions.
We characterize the oracle complexity of finding first-order stationary points in smooth nonconvex--strongly-concave (NC--SC) minimax optimization when each query provides the full Hessian in addition to the function value and gradient. For $\Phi(x)=\max_y f(x,y)$, assume that $f$ has an $L$-Lipschitz gradient, a $\rho...
Fang-Qing Zhang, Hai-Han Zhang, Wen-Dao Wu et al.· 0 citations
Does randomization improve the worst-case first-order complexity of nonconvex optimization when the objective is very smooth? We prove that, for every fixed finite integer $p\ge3$, finding an $\eps$-stationary point requires $\Omega_p(\eps^{-5/3})$ exact value-and-gradient queries, even for $C^\infty$ objectives with i...
Hai-Han Zhang, Wen-Dao Wu, Fang-Qing Zhang et al.· 0 citations
A proper, full-information learner on compact convex subsets of $\mathbb{R}^d$ that adapts to the unknown path length and, under fixed dimension-independent diameter, gradient, and curvature parameters, guarantees full information under fixed dimension-independent diameter, gradient, and curvature parameters.
Yan-Yi Li, Wen-Dao Wu, Hai-Han Zhang et al.· 0 citations
We study the sample complexity of finding a point with expected hypergradient norm at most $\epsilon>0$ in nonconvex--strongly-convex stochastic bilevel optimization using fresh, globally unbiased first-order samples with bounded variance. For lower-variable smoothness order $p\ge1$, F$^2$SA-$p$ achieves $\widetilde O(...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
We study deterministic adaptive optimization of globally $\beta$-smooth, $\mu$-strongly convex functions using exact scalar function values. Queries and outputs lie in $B_2^d(R)$, and the minimizer lies in $B_2^d(R/2)$. Set $\kappa=\beta/\mu$, $Q=\beta R^2/\epsilon$, and $D_d=(d/\log(ed))^{1/3}$. For sufficiently large...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
We establish a randomized first-order lower bound for finding an $\epsilon$-stationary point, $\|\nabla f(x)\|\le \epsilon$, of a nonconvex function with initial gap at most $\Delta$, $L_1$-Lipschitz gradient, and $L_2$-Lipschitz Hessian. Each oracle call returns the exact function value and gradient. Let $Q_{\mathrm{r...
We prove a joint stochastic zeroth-order lower bound for Goldstein stationarity on a Euclidean query ball, even when the ball is guaranteed to contain a stationary point. In dimension $d$, let $f=\mathbb{E}[F(\cdot;\xi)]$, assume $\mathbb{E}[\operatorname{Lip}(F(\cdot;\xi))^2]\le L_0^2$, and bound the initial objective...