This work introduces a novel notion of strong convexity, termed Restricted Relative Strong Convexity, and establishes linear convergence rates for BPGM under this condition, and exploits the proposed theoretical framework to provide an in-depth analysis of the convergence of BPGM for (regularized) Kullback--Leibler regression problems.
Abstract
Bregman Proximal Gradient methods (BPGM) exploit the underlying geometry of the objective function through a carefully chosen mirror map. In this work, we introduce a novel notion of strong convexity, termed Restricted Relative Strong Convexity, and establish linear convergence rates for BPGM under this condition. We then exploit the proposed theoretical framework to provide an in-depth analysis of the convergence of BPGM for (regularized) Kullback--Leibler regression problems, covering scenarios with both unique and non-unique minimizers, as well as regularized and unregularized formulations. Specifically, we demonstrate that using the popular Burg's entropy as a distance-generating function may only yield linear convergence for certain KL regression problems. In contrast, we show that employing a smoothed version of the Burg's entropy induces the suitable geometry required to guarantee linear convergence. We conclude with numerical experiments that nicely align with our theoretical findings.
A novel convergence analysis framework for the BPGM with the Shannon entropy kernel is developed, yielding strong convergence results for a broad class of objective functions under linear constraints.
Ada-BPSG is introduced, a line-search-free BPSG method that couples the SAGA gradient table with a stabilized Barzilai--Borwein (BB) candidate and yields a direct analytical chain from relative smoothness and component-wise variance control to convergence in finite-dimensional normed spaces.
Chenhan Jin, Shengze Xu, Binghui Xie et al.· 0 citations
This study presents a novel Deep Proximal Gradient Descent framework for ill-posed problems by employing a tailored second-order differentiable Input-Convex Neural Networks (ICNNs) as a learned regularizer. A key contribution is the design of convex residual mapping, which preserves the convexity of the regularized objective, thereby enhancing the interpretability of the deep network without sacrificing its expressive power. Based on this framework, we develop two types of algorithms. For linear problems, the ICNN-based regularizer is embedded into the standard proximal gradient structure. For nonlinear problems, we introduce an innovative formulation that employs the learned residual to guide gradient descent, while using the traditional data misfit as a proximal regularizer to avoid network-dominated spurious solutions. Building on this iterative scheme, we establish groundbreaking convergence results for both algorithms, complete with rigorous proofs. Extensive numerical experiments, particularly on real low-dose Computed Tomography data, validate the superior imaging quality and high computational efficiency of our algorithms.
T. Ye, Guangyu Gao, Yang Li et al.· Inverse Problems· 0 citations
This work can specifically ensure, without any smoothness assumptions, convergence to Mordukhovich stationarity as long as the base directions asymptotically revert to the negative gradient for small stepsizes.
This work develops comparison-oracle variants of Normalized Gradient Descent and Gradient Descent with Polyak stepsizes and establishes explicit upper bounds on the approximation error that guarantee convergence and derive convergence rates for all proposed methods.
A globally convergent regularized Newton method with positive definite regularization for solving nonsmooth optimization problems that replaces the identity matrix in traditional algorithms with a general positive-definite symmetric matrix to regularize the generalized Hessian.
