Sequence convergence to a boundary Karush--Kuhn--Tucker (KKT) point has long remained unclear for nonconvex mirror descent with Legendre kernels. The difficulty arises from the blow-up of the gradient of the Legendre kernel at the boundary. Recent work~\cite{dingtoh2026nonkkt} shows that mirror descent can accumulate at non-KKT boundary points despite decreasing objective values, precluding a convergence guarantee to KKT points in general. Despite this negative result, mirror descent remains effective in many real applications. Motivated by this contrast, we address the boundary difficulty directly and establish KKT convergence of mirror descent for a broad class of structured nonconvex problems. We analyze mirror descent in reparameterized variables, where the Hessian metric is flattened and remains nondegenerate as the boundary is approached. Under extension and definability conditions jointly coupling the objective, the Legendre kernel, and the feasible region, the reparameterized sequence has finite length and converges, thereby recovering convergence to a KKT point of the original sequence. Our general framework applies to some concrete instances: Shannon entropy, Fermi--Dirac entropy, and power kernels on polyhedron.
For mirror descent generated by a Legendre kernel, perhaps one of the most basic question in optimization is this: must every accumulation point of a bounded mirror descent sequence be Karush--Kuhn--Tucker (KKT) stationary under proper stepsizes? We show that the answer is no. A longstanding obstacle to resolving this question is the boundary blow-up of the Legendre gradient: it keeps every mirror step in the interior, while at a boundary limit, the inverse entropy metric vanishes on active coordinates and can erase the dual-feasibility in the KKT system. We construct $C^\infty$ objectives and bounded sequences generated by the Shannon-entropic mirror descent on the nonnegative orthant $\R_+^n$, for every $n\geq 3$, and on the probability simplex $\Delta_n$, for every $n\geq 4$, such that, in each case, the set of accumulation points is a smooth boundary circle containing a nonempty relatively open arc of non-KKT points. The steps satisfy $\alpha_k\asymp k^{-\beta}$ with $\beta\in(1/2,1)$, the objective values are nonincreasing, and the objectives are entropy-relatively smooth. Hence the pathology stems from the degeneracy of the Bregman geometry at the boundary, rather than from failure of descent, or improper stepsizes. To the best of our knowledge, these provide the first counterexamples to KKT accumulation for bounded mirror descent sequences with nonincreasing objective values.
We analyze Bregman ADMM for nonconvex linearly constrained problems under two-sided relative smoothness, a condition that replaces the standard Lipschitz gradient assumption with a Hessian comparison relative to a Bregman kernel. This setting covers polynomial objectives arising in matrix and tensor models for which a global Lipschitz-gradient constant need not exist. We show that on an invariant open state-space domain, one iteration of Bregman ADMM defines a smooth primal--dual fixed-point map whose strict-saddle KKT points are unstable fixed points; consequently, from random initialization the iterates converge to a strict saddle with probability zero. Combined with existing first-order convergence results, this yields almost-sure second-order stationarity of limiting KKT points. We extend the analysis to a multi-block star consensus formulation for distributed optimization. The technical novelty lies in a determinant reduction with a Bregman-specific symmetrization and scaling step in the two block spectral argument, together with a null space cancellation exploiting the star graph structure in the consensus case. Numerical experiments on distributed matrix factorization illustrate the theory, and a symmetric tensor factorization example demonstrates the broader Bregman proximal splitting idea beyond the separable consensus setting.
S. Li, Zhihui Zhu, Qiuwei Li· arXiv.org· 0 citations
First-order methods typically require a specific step-size that depends on the regularity conditions of the objective function, such as the smoothness, Lipschitz continuity, or strong convexity constants. The Polyak step-size is a classical alternative for subgradient descent on convex functions that only uses knowledge of the optimal value of the objective function and automatically adapts to the above-mentioned regimes. However, many optimization problems are better described by non-Euclidean geometries and are more amenable to mirror descent. Extending this adaptivity to mirror descent is subtle. Some existing generalizations of the Polyak step-size rely on norms instead of purely on relative geometry, excluding many of the use cases of mirror descent. In this work, we revisit a variant of the Polyak step-size based on Bregman projections due to Kiwiel (1997), which we call mirror Polyak. This method is known to converge asymptotically, but its convergence rate is not known. We show that mirror Polyak enjoys guarantees similar to its Euclidean counterpart, automatically adapting to relative notions of smoothness, Lipschitz continuity, or strong convexity. We then leverage mirror Polyak to avoid having to know the optimal value in some structured optimization problems such as regularized linear and logistic regression. We propose a lifted formulation based on convex duality with optimal value exactly zero and a natural mirror map given by the problem's structure. Mirror Polyak applied to the lifted problem enjoys the same worst-case guarantees as the Polyak step-size in the original problem if we knew the optimal value.
Frederik Kunstner, Ryan D'Orazio, V. S. Portella et al.· 0 citations
Nonlinear optimization problems with complicated, nonconvex, yet geometrically structured constraints can be tackled by projected-gradient methods: under weak regularity assumptions, these approaches were recently proved to possess convergence properties to the strongest stationarity conditions. In this work, we show how momentum terms, commonly used in nonlinear optimization to speed up the convergence process, can be integrated within this algorithmic framework without harming convergence guarantees. Preliminarily, we highlight an intrinsic issue induced by the direct replacement of the negative gradient with a general descent direction within the projected approach. Then, we present suitable backtracking mechanisms for the pre-projection step, allowing us to integrate momentum terms in the direction. By this technique, we can specifically ensure, without any smoothness assumptions, convergence to Mordukhovich stationarity as long as the base directions asymptotically revert to the negative gradient for small stepsizes; moreover, if the base search direction reverts exactly to the negative gradient for the smallest steps, the algorithm is proved to converge to Bouligand and Proximally stationary points, with and without (local) smoothness assumptions respectively. Finally, the proposed procedure is numerically tested on some classes of problems, namely, sparsity and bounded-rank constrained problems; the results indicate that the proposed method is computationally effective, taking advantage of the additional information provided by the momentum term.
The iterate convergence of \textit{Bregman projected gradient method} (BPGM) has remained a long-standing open problem, especially for the widely adopted Shannon entropy kernel. Existing convergence results are often limited, relying on Lipschitz continuity of the kernel's gradient or restrictive conditions on the objective function. In this paper, we develop a novel convergence analysis framework for the BPGM with the Shannon entropy kernel, yielding strong convergence results for a broad class of objective functions under linear constraints. The cornerstone of our framework is a new concept called \textit{scaled Kurdyka-\L{}ojasiewicz} (SK\L{}) property, which captures the local growth behavior of a function under the Bregman geometry. We show that the SK\L{} property ensures the iterate convergence of BPGM and holds for all continuous subanalytical functions. Furthermore, we prove that the BPGM sequence exhibits linear convergence if the problem possesses an SK\L\ exponent of $1/2$. We then furnish the examples of functions with the SK\L\ exponent $1/2$ by proving that the SK\L\ exponent $1/2$ is implied by the K\L{} exponent $1/2$ under strict complementarity and local Lipschitz continuity of the objective's gradient. Building on these novel results, our work takes a first step towards resolving the open problem of BPGM iterate convergence.
In this paper, we study dual semismooth Newton (SSN) methods for degenerate polyhedral projection problems, where generalized Jacobians of the dual residual may remain singular even arbitrarily close to the solution set. Rather than regularizing these singular systems, we exploit the nonuniqueness of the dual representation. We introduce a primal--dual lifted projection-equivalent set that always possesses extreme points without additional structural assumptions on the polyhedron, and show that its extreme-point geometry identifies dual representatives at which nonsingular generalized Jacobians of the dual residual can be constructed. This geometry is further linked to a full-column-rank condition and a generalized weak strict Robinson constraint qualification, showing that the regularity required by the Newton step can be recovered rather than imposed \emph{a priori}. We also establish displacement bounds that connect representative selection throughout the algorithm with the local Newton mechanism. Building on this variational framework, we develop an inexact dual SSN method with local superlinear convergence and a globalized version combining monotone representative selection with a Wolfe line search. The resulting method is globally convergent and eventually recovers the fast local rate. Numerical experiments on regularized optimal transport, battery-scheduling feasibility restoration, and occupation-measure projection demonstrate its robustness in highly degenerate settings.