This work analyzes a stochastic Newton optimization scheme for locating the unique global minimizer of a general nonconvex objective function and proves global almost sure convergence.
Abstract
We analyze a stochastic Newton optimization scheme for locating the unique global minimizer of a general nonconvex objective function. The method couples a Newton algorithm to additive Gaussian noise with state-dependent variance. In the bounded domain setting, we prove global almost sure convergence. The proof is based on two features of the algorithm: a nondegenerate exploratory property that ensures entrance into a neighborhood of the minimizer after a finite number of steps, and a decaying-noise property that yields contraction with high probability and prevents infinitely many exits from the neighborhood of the minimum.
We investigate the optimization problem of minimizing a nonsmooth function that satisfies a nonsmooth version of the descent lemma over a nonempty and closed but not necessarily convex set. The objective function belongs to the class of upper-$\mathcal{C}^2$ functions, whereas the constraints may promote a sparse or lo...
Christian Kanzow, Jannis Krüger, Leo Lehmann· 0 citations
In the nonconvex case, it is proved that quadratic growth in conjunction with a regularity property for the active gradients suffices, and it is proved that quadratic growth in conjunction with a regularity property for the active gradients suffices.
For solving nonconvex equality-constrained optimization problems, a recent Gradient-Eigenstep Algorithm by Goyens et al.~is an iteration-efficient approach, based on minimizing Fletcher's augmented Lagrangian function, for finding an approximate second-order stationary point from an arbitrary starting point. In this pa...
F. Curtis, Ling-Jun Guo, Daniel P. Robinson· 0 citations
A new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian that achieves the global convergence rate of $O(1/k^3)$ in terms of the functional residual, the first second-order method for this problem class attaining this rate.
A zeroth-order stochastic proximal Newton-type algorithm with a new norm test for adaptive sample size selection is proposed and both iteration complexity and oracle complexity of the algorithm are established for nonconvex and strongly convex objectives, respectively.
Ze-Kun Liu, Jinyan Fan· Journal of Scientific Comput...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.