This work develops a regularization analysis of non-accelerated and accelerated primal-dual methods for solving linear inverse problems in the presence of noisy data. We investigate a Condat-V\~u algorithm and an accelerated primal-dual hybrid gradient method in Hilbert spaces, with focus on quantifying the effect of data perturbations on the reconstruction error. For the non-accelerated scheme, we derive error estimates in terms of Bregman distances, whereas for the accelerated scheme we establish error estimates in norm. The study accommodates a general class of convex data fidelities satisfying suitable perturbation conditions, which are verified explicitly for equality constrained and Morozov regularization. For the non-accelerated method, the analysis is further extended to Banach spaces, taking into account non-Euclidean geometries and including a particular non-reflexive setting tailored to nonnegative solution reconstruction. The results recover known behavior in classical settings while extending the regularization analysis to these more general frameworks. Numerical experiments with representative regularizers, including sparsity and entropic models, support the theoretical findings and illustrate practical performance under noise.
A globally convergent regularized Newton method with positive definite regularization for solving nonsmooth optimization problems that replaces the identity matrix in traditional algorithms with a general positive-definite symmetric matrix to regularize the generalized Hessian.
We study Langevin-based methods for non-convex optimization under smoothness and dissipativity assumptions. Our focus is on obtaining non-asymptotic bounds for the expected excess risk rather than sampling guarantees for the full target distribution. The key ingredient of our analysis is a direct passage from relative entropy to objective-value error, based on a weighted Csisz\'ar--Kullback--Pinsker inequality and exponential-moment estimates. This avoids intermediate Wasserstein bounds and yields sharper dependence on the Log-Sobolev constant, a quantity that may scale exponentially with the inverse temperature and the dimension in non-convex problems. We first analyze the Unadjusted Langevin Algorithm with exact gradients and derive explicit bounds on $\mathbb{E}[F(x_k)]-\min F$ in terms of the inverse temperature, dimension, stepsize, smoothness and dissipativity parameters, and the Log-Sobolev constant. We then extend the result to an inexact-gradient version of ULA, allowing for biased and stochastic gradient surrogates whose mean-square error grows at most quadratically in the state. This framework covers stochastic gradients and zeroth-order estimators based only on function evaluations. In particular, we show that both Gaussian and spherical finite-difference estimators fit into the inexact-ULA theory and obtain explicit function-evaluation complexity bounds for zeroth-order Langevin optimization. To the best of our knowledge, these are the first non-asymptotic global non-convex optimization complexity bounds for zeroth-order ULA. We also provide numerical experiments illustrating the behavior of the proposed zeroth-order Langevin schemes.
E. Naldi, Marco Rando, Lorenzo Rosasco et al.· 0 citations
It is shown that strict complementarity, together with a quadratic facial-violation property of the associated complementary faces, implies uniform quadratic growth of both the primal and dual augmented Lagrangians near a strictly complementary solution, and the local equivalence of three regularity conditions is proved.
In this paper, we study a Tikhonov-regularized mixed-order primal--dual dynamical system with implicit Hessian damping for linearly constrained convex optimization problems in finite-dimensional Euclidean spaces, where the primal equation is second order and incorporates the viscous damping term \(\delta\sqrt{\varepsilon(t)}\,\dot x(t)\), whereas the multiplier equation remains first order. By constructing a new class of energy functions, for a general Tikhonov regularization coefficient \(\varepsilon(t)\), we prove the strong convergence of the primal trajectory and derive fast convergence rates under the same parameter assumptions, without imposing any eventual inside/outside-ball condition. More precisely, the primal trajectory converges to the minimum-norm solution, and the multiplier converges to a compatible KKT multiplier, while the convergence rates of the Lagrangian gap, feasibility violation, and objective residual are \(o(\varepsilon(t))\), and the convergence rate of the velocity norm is \(o(\sqrt{\varepsilon(t)})\). For the critical case \(\varepsilon(t)=c/t^2\), in which the damping coefficient \(\delta\sqrt{\varepsilon(t)}\) reduces to \(\delta\sqrt{c}/t\), we establish the sharper convergence rates \(o(t^{-2})\) for the Lagrangian gap, feasibility violation, and objective residual, together with \(o(t^{-1})\) for the velocity norm, which improve the corresponding \(O(t^{-2})\) and \(O(t^{-1})\) decay estimates obtained in the related literature. Most importantly, when the proposed dynamical system is specialized to the finite-dimensional unconstrained setting, our analysis answers the open question on strong convergence in this critical regime posed by Attouch and L\'aszl\'o [Math. Methods Oper. Res., 99 (2024), pp.~307--347].
Stochastic gradient methods have gained increasing attention for solving large-scale inverse problems due to their computational efficiency. However, their theoretical justification in the context of ill-posed problems remains underdeveloped, particularly regarding convergence rate analysis, where existing results typically yield only suboptimal rates. In this paper, we address this gap by establishing order-optimal convergence rates for a stochastic gradient method applied to linear ill-posed problems in Hilbert spaces. Under Hölder-type source conditions with smoothness parameter
$$\nu \in (0, 1/2]$$
ν
∈
(
0
,
1
/
2
]
, we derive convergence rates both in expectation and almost surely, accommodating a broad class of step-size sequences, including constant and polynomially decaying ones. Our analysis is based on a delicate Lyapunov-type argument and an application of the Robbins–Siegmund theorem. As a byproduct, we also establish new convergence results that do not rely on any source conditions.
Qi-Nian Jin, Xi-Liang Lu, Lin Tian· Numerische Mathematik· 0 citations
Empirical results reveal that VR-DR remains highly effective for nonsmooth loss functions, significantly broadening its practical utility beyond its theoretical constraints.
Zehui Jia, Deng-Hui Li, Zhiyu Liu et al.· Journal of Scientific Comput...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
