Research on Neural Network Training Mechanism Integrating Convex Optimization and Backpropagation
TL;DR
This paper deeply integrates convex optimization theory with the backpropagation algorithm and constructs a novel stable and efficient training mechanism for neural networks that achieves favorable adaptability to both shallow fully connected networks and deep convolutional networks.
Abstract
Traditional neural network training based on backpropagation suffers from multiple bottlenecks, including slow convergence rate, susceptibility to local optima, vanishing/exploding gradients, and insufficient generalization performance. To address these issues, this paper deeply integrates convex optimization theory with the backpropagation algorithm and constructs a novel stable and efficient training mechanism for neural networks. Systematical optimization of the conventional training pipeline is realized via convex reconstruction of the loss function, design of an adaptive gradient correction rule under convex optimization constraints, and rigorous theoretical proof of convergence for the integrated algorithm. Experimental results demonstrate that compared with mainstream algorithms such as standard BP, SGD and Adam, the proposed mechanism reduces the number of convergence iterations by over 35%, cuts training time by 28%, improves classification accuracy by 4%-7%, and effectively suppresses gradient anomalies. It achieves favorable adaptability to both shallow fully connected networks and deep convolutional networks. This research complements the theoretical convex optimization framework for non-convex training, and provides methodological support and theoretical references for efficient training and industrial deployment of deep learning models.