Conference

Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training

Anxhelo Shehu Enes Stastoli Arben Çela

Jul 2026 · International Conference on Control, Decision and Information Technologies · pp. 1619-1625 · 0 citations · 23 references

Computer Science

TL;DR

Indicative in these results is that intelligent batch ordering can partially compensate for reduced architectural complexity, thus offering a computationally efficient alternative to deeper models.

Abstract

Common practice when training Convolutional Neural Networks (CNNs) is to utilize randomly shuffled mini-batches. During this stage of the training process, two limitations are faced. Firstly the slowing down of convergence, and secondly the limited learning signal contribution due to large quantities of samples that are considered easy while training CNNs.In our work, we aim to address these inefficiencies by proposing A*-Inspired Batch Selection (A*-BS). This is a lightweight and model-agnostic training strategy that formulates mini-batch scheduling as a heuristic search problem. In this type of batch selection process each batch is treated as a node in a search space and ranked using an A*-like score by combining a loss-based difficulty measure with a reuse penalty. The two main benefits of this method are: informative gradient updates encouragement and batch diversity selection throughout the training process. The proposed method does not modify network architectures or optimization algorithms and can be seamlessly integrated into existing training pipelines. We evaluate the approach on the twelve 2D classification tasks of the MedMNIST-v2 benchmark, using a deliberately simple architecture of approximately 2.25 × 105 parameters and comparing against the ResNet-18 and ResNet-50 baselines reported by the benchmark.On half of these tasks, the lightweight model combined with A*-BS reaches higher accuracy and AUC than both ResNet baselines, with relative gains of up to 15%. Moreover, an ablation under identical CNN architecture and hyper-parameters shows that A*-BS outperforms random batch shuffling on all twelve MedMNIST tasks. Wall-clock measurements further show that the lightweight CNN with A*-BS trains substantially faster than ResNet-18 and ResNet-50 on identical hardware.Indicative in these results is that intelligent batch ordering can partially compensate for reduced architectural complexity, thus offering a computationally efficient alternative to deeper models. Furthermore, the reliability of the proposed strategy is reinforced by its strong performance with a simple CNN, even when compared to deeper and more sophisticated architectures.

DOI arXiv Semantic Scholar

View source

Similar papers

Training to Design Convolutional Neural Networks

It is demonstrated that network architecture and its coeﬁcients can be learned together by unifying concepts of evolutionary search within a population based traditional training process.

D. Sapra, A. Pimentel · 0 citations

Open access 2026

Boosting Lightweight CNN-Based Networks Via Selective Residual Attentive Patterns for Image Recognition

Modern deep networks often rely on attention modules, which are still at a modest level due to using either one type of channel-wise pattern or an expensive combination of two types of them. In the case of using all of those, the obtained weights can be less discriminative due to the disjointed excitations, while the model complexity would double. To deal with these limitations, an efficient attention is proposed by addressing two novel components of residual attentive information as follows: 1) top- $n$ channel-residual attentive patterns with a unitary excitation perceptron, and 2) multiple spatial-residual attentive features. A simple fusion of these complementary components forms a robust volume of selective residual attentive patterns (named SRAP). Experiments on benchmark datasets for image classification have proved the prominent performance of SRAP versus other attention modules. Particularly, SRAP boosted the performance of lightweight CNN-based networks by up to ~7% on ImageNet-100 without increasing the computational complexity. The implementation code of SRAP is available at https://github.com/nttbdrk25/SRAP.

Thanh Tuan Nguyen, Hoang Anh Pham, Thinh Le Vinh et al. · IEEE Access · 0 citations

Open access Aug 2026

Cross-Architecture Assessment of Hyperparameter Optimization Techniques in Convolutional Neural Networks

It is demonstrated that hyperparameter optimization dynamics depend heavily on dataset complexity, where computational efficiency is the primary differentiator for simpler classification tasks, but optimization architecture selection becomes critical for navigating challenging medical imaging applications.

Sarab Almuhaideb, Ahmad Raza Khan · Applied Sciences · 0 citations

Aug 2026

Reinforcement Learning Pruning with Dynamic Reward Function and Joint Selecting Mechanism

A dynamic pruning method that uses reinforcement learning guided by a dynamic reward function to automatically explore the pruning rate of each layer, and uses the designed similarity-based connected graph (SCG) mechanism to perform channel removal operations during training.

Ruisi Cai, Yixin Zhao · Machine-mediated learning · 0 citations

#machine learning Preprint Aug 2026

Residual-Guided Randomized Neural Networks

A simple and broadly applicable residual guided procedure that greedily constructs the hidden layer using a closed form residual decrease criterion and yields a progressive training process with a guaranteed monotonic decrease of the training objective.

M. Akhtar, M. Tanveer, Mohd. Arshad · 0 citations

Preprint Aug 2026

Designing Compact Neural Architectures via Neuron Gating and Mixed Activation

This work develops a general bilevel optimization framework for NAS across diverse architectures, including MLPs, CNNs, RNNs, and Transformers, to identify compact architectures with strong predictive performance.

Abhishek Shukla, Ankur Sinha, Faiz Hamid · 1 citation