Scalable Levenberg-Marquardt Training Framework for Deep Learning
The Levenberg-Marquardt (LMA) algorithm is a widely used nonlinear least-squares solver known for its rapid convergence. For small-scale neural networks, LMA provides better convergence and architectural robustness than first-order optimizers like Adam. However, LMA imposes prohibitive memory and computational costs. T...
Xiang-Hao Wang, Johan Barthélemy, John A. Taylor
· 0 citations