It is demonstrated that hyperparameter optimization dynamics depend heavily on dataset complexity, where computational efficiency is the primary differentiator for simpler classification tasks, but optimization architecture selection becomes critical for navigating challenging medical imaging applications.
In this paper, we perform a systematic empirical study of deep learning techniques on the CIFAR-10 image classification benchmark. We study three inter-related aspects of neural network design: (1) the relative impact of nine gradient descent optimisation algorithms on a baseline Multi-Layer Perceptron (MLP); (2) the effectiveness of six regularisation techniques namely; L2 weight decay, Dropout, Data Augmentation, Gaussian Noise Injection, Early Stopping, and Ensemble Learning; on model generalisation; and (3) a benchmark comparison of five landmark Convolutional Neural Network (CNN) architectures alongside Inception V3 trained from scratch. We also investigate autoencoder-based image reconstruction and a side-by-side comparison of linear Principal Component Analysis (PCA) with non-linear autoencoder representations. Nesterov Accelerated Gradient achieves the lowest training loss (1.5814) in short-run experiments. VGGNet achieves the highest test accuracy (75.09%) among CNN architectures trained from scratch for five epochs. Early Stopping yields the best regularisation outcome, and non-linear autoencoders outperform PCA in reconstruction quality. All experiments use TensorFlow 2.x and Keras on CIFAR-10.
HIFN-T is presented, a framework extending the Variational Information Bottleneck through four jointly learnable per-layer parameters: information retention, entropy budget, magnitude scaling, and global information gates that generalizes standard VIB as a special case and characterize the role of the entropy budget as an early-training stabilizer that prevents variance explosion before convergence.
Recent advancements in deep reinforcement learning have increasingly favored simplified, highly parallelized paradigms. Notably, the Parallelized Q-Network (PQN) algorithm enables off-policy value learning without relying on experience replay buffers or target networks. However, the representational capacity and computational efficiency of visual encoders operating in these buffer-free settings remain comparatively underexplored. In this work, we systematically investigate the architectural design space of Convolutional Neural Networks within PQN. We evaluate eight distinct CNN topologies while explicitly characterizing their parameter and computational requirements. We further study the effect of multiplicative representation learning and advanced value estimation by integrating the Hadamax encoding paradigm with categorical, ensemble, and dueling value heads. Extensive experiments on Atari-57 show that our final composite architecture, Aftab, achieves an Interquartile Mean (IQM) Human-Normalized Score of 6.592, compared with 2.715 for the standard PQN baseline, together with a 0.86 Probability of Improvement over PQN. We additionally evaluate Aftab on Procgen-Hard to assess performance under procedurally varying visual environments. Aftab achieves a normalized learning-curve Area Under the Curve (nAUC) of 0.541 compared with 0.216 for PQN. Overall, the results demonstrate that carefully designed encoder topology, multiplicative feature interactions, and advanced value-estimation heads can substantially improve performance within a parallelized, replay-free Q-learning framework while preserving its memory-efficient training paradigm. The complete Aftab framework, including model definitions, training configurations, reproducibility settings, and raw experimental logs, is open-sourced at https://github.com/tahashieenavaz/aftab
Modern deep networks often rely on attention modules, which are still at a modest level due to using either one type of channel-wise pattern or an expensive combination of two types of them. In the case of using all of those, the obtained weights can be less discriminative due to the disjointed excitations, while the model complexity would double. To deal with these limitations, an efficient attention is proposed by addressing two novel components of residual attentive information as follows: 1) top- $n$ channel-residual attentive patterns with a unitary excitation perceptron, and 2) multiple spatial-residual attentive features. A simple fusion of these complementary components forms a robust volume of selective residual attentive patterns (named SRAP). Experiments on benchmark datasets for image classification have proved the prominent performance of SRAP versus other attention modules. Particularly, SRAP boosted the performance of lightweight CNN-based networks by up to ~7% on ImageNet-100 without increasing the computational complexity. The implementation code of SRAP is available at https://github.com/nttbdrk25/SRAP.
Continuous advancements of deep learning techniques have profoundly influenced Artificial Intelligence (AI) for visual classification through shifting the field from manual feature engineering to autonomous, hierarchical feature learning. On the contrary, the traditional mechanisms for visual classification relied on various challenges, including large data requirements, computational demands, model interpretability issues, and bias concerns, which severely limited accurate classification. Therefore, the research proposes the Fight-Or-Free Optimized Distributed Patch-Wise Attention-Driven Bidirectional Long Short-Term Memory Network (F2DPAB-Net) for visual classification. The Fight-Or-Free Optimization (F2Opt) algorithm significantly tunes the hyperparameters using stochastic behaviors, potentially improving convergence speed and providing a balance between local exploitation as well as global exploration. Integration of patch-wise triplet attention fusion mechanism offers parallel processing, making the model more efficient in learning long-range dependencies that substantially increase the significance while training. On top of that, utilization of multimodality features enables the model to process and understand different modalities that achieve a more comprehensive interpretation of information and improve generalization. Overall, the proposed F2DPAB-Net outperforms existing methods, thus attaining a maximum of 0.981 Cohen’s Kappa Score, 0.96 MCC, and 0.984 NPV using the COCO dataset, respectively.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
