Building a Neural Network from Scratch: Implementation, Evaluation, and Optimization

LEMUR 2: Unlocking Neural Network Diversity for AI

This dataset defines a new basis for reproducible and data-driven AI design, advancing the emerging paradigm of LLM-driven AutoML and architectural generalization across modalities and hardware.

History, Development, and Principles of Representation Learning—An Introductory Survey

This survey deeply explains the basic principles of representation learning, and introduces its practical application cases in various fields, and points out the main limitations of current models and prospects the future research directions.

Simplifying Neural Networks During Training

Understanding and exploiting the training dynamics of overparameterized deep neural networks remains a central challenge in modern machine learning. Recent evidence on Neural Collapse (NC) shows that class representations and classifiers exhibit highly structured geometry, while the Tunnel Effect suggests that only a subset of layers is essential for feature extraction. We combine these two perspectives and propose an NC-inspired training framework for simplifying deep networks during training. Our method monitors representation dynamics through the Inverse Fisher Criterion, a stable and efficient proxy for the variability collapse behavior, to identify both the split point between feature extraction and classification and the training stage at which simplification becomes viable. We then replace the trailing layers with a lightweight classification head and continue training the reduced model. Experiments on image-classification benchmarks across MLP, VGG, and ResNet architectures show that the proposed method achieves substantial parameter reductions while maintaining accuracy comparable to that of the full model. Code to reproduce the experiments can be found at: https://github.com/LorenzoSciandra/NNS.

Deep Learning Enhancements Using Pretraining and Fine-Tuning

Experimental findings show that pretrained models outperform those trained from scratch in terms of accuracy, convergence speed, and robustness, and a unified framework is proposed to integrate both processes in a deep learning pipeline.

Training to Design Convolutional Neural Networks

Research on Neural Network Training Mechanism Integrating Convex Optimization and Backpropagation

Traditional neural network training based on backpropagation suffers from multiple bottlenecks, including slow convergence rate, susceptibility to local optima, vanishing/exploding gradients, and insufficient generalization performance. To address these issues, this paper deeply integrates convex optimization theory with the backpropagation algorithm and constructs a novel stable and efficient training mechanism for neural networks. Systematical optimization of the conventional training pipeline is realized via convex reconstruction of the loss function, design of an adaptive gradient correction rule under convex optimization constraints, and rigorous theoretical proof of convergence for the integrated algorithm. Experimental results demonstrate that compared with mainstream algorithms such as standard BP, SGD and Adam, the proposed mechanism reduces the number of convergence iterations by over 35%, cuts training time by 28%, improves classification accuracy by 4%-7%, and effectively suppresses gradient anomalies. It achieves favorable adaptability to both shallow fully connected networks and deep convolutional networks. This research complements the theoretical convex optimization framework for non-convex training, and provides methodological support and theoretical references for efficient training and industrial deployment of deep learning models.