Aug 2026· International journal of computer information systems and industrial management applications· 0 citations
TL;DR
A novel adaptive fusion framework that adaptively combines CNN and Transformer features through learnable gating, attention-based feature integration, and explainable-AI methods is developed, intended to improve both computational efficiency and model interpretability.
Abstract
Over the past few years, deep learning has changed substantially following the emergence of Transformer architectures, which are particularly effective for representing long-range dependencies that are difficult for conventional Convolutional Neural Networks (CNNs). Whereas CNNs are well suited to extracting local spatial features using convolutional operations, Transformers are effective at representing global context through self-attention. Hybrid CNN–Transformer architectures have been developed to combine the respective strengths of the two approaches. A limitation of many existing models is their reliance on static or manually designed fusion strategies, which can restrict adaptability, add computational cost, and make the resulting decisions harder to interpret. The present study develops a novel adaptive fusion framework that adaptively combines CNN and Transformer features through learnable gating, attention-based feature integration, and explainable-AI methods. The resulting framework is intended to improve both computational efficiency and model interpretability, thereby addressing important limitations of current hybrid designs. The experimental evaluation uses benchmark datasets such as ImageNet, CIFAR-100, and medical imaging datasets. The reported results show that the proposed model performs better than the comparison architectures with respect to accuracy, efficiency, and interpretability.
Modern deep networks often rely on attention modules, which are still at a modest level due to using either one type of channel-wise pattern or an expensive combination of two types of them. In the case of using all of those, the obtained weights can be less discriminative due to the disjointed excitations, while the model complexity would double. To deal with these limitations, an efficient attention is proposed by addressing two novel components of residual attentive information as follows: 1) top- $n$ channel-residual attentive patterns with a unitary excitation perceptron, and 2) multiple spatial-residual attentive features. A simple fusion of these complementary components forms a robust volume of selective residual attentive patterns (named SRAP). Experiments on benchmark datasets for image classification have proved the prominent performance of SRAP versus other attention modules. Particularly, SRAP boosted the performance of lightweight CNN-based networks by up to ~7% on ImageNet-100 without increasing the computational complexity. The implementation code of SRAP is available at https://github.com/nttbdrk25/SRAP.
Thanh Tuan Nguyen, Hoang Anh Pham, Thinh Le Vinh et al.· IEEE Access· 0 citations
TransCat, a hybrid CNN-Transformer architecture for medical image segmentation, is proposed and an extended deformable attention mechanism with attentive value identification is developed, to control the computational burden caused by the enlarged token set.
Jin Wang, Zheng-Hua Yang, Dong-Ming Zhou et al.· Frontiers in Bioinformatics· 0 citations
Automated diabetic retinopathy grading using fundus images demands accurate, computationally efficient, robust, and interpretable models. Although convolutional neural networks are effective at extracting local lesion patterns, they have limited ability to model long-range spatial relationships across the retina, while standard vision transformers provide global context at a substantially higher computational cost. To address this trade-off, this study proposes a lightweight hybrid CNN–Transformer framework that combines an EfficientNet-B0 backbone for local retinal feature extraction with a compact four-layer transformer branch for global contextual modeling. The final EfficientNet-B0 feature map is con-verted into spatial tokens and processed by the transformer, after which CNN and transformer representations are inte-grated through a learned feature-fusion module. The model contains only 6.2 million parameters and achieved strong five-class DR grading performance on APTOS 2019, with a quadratic weighted kappa of 0.920 under stratified five-fold cross-validation. External testing on MESSIDOR-2 further showed promising cross-dataset generalization, achieving 94.7% binary screening accuracy and a QWK of 0.891 without fine-tuning. In addition, Grad-CAM and attention rollout were used to provide complementary local and global explanations of model predictions. The results indicate that the proposed framework presents a concise and semantically meaningful method for DR grading and may facilitate future clinical decision-support assessments, but future prospective validation with ophthalmologists re-mains required before its implementation in a real clinical setting.
Rasha Jamal Hindi· Muthanna Journal of Engineer...· 0 citations
The results show that larger models and larger pretraining datasets do not automatically lead to better downstream performance, and transfer effectiveness in medical imaging is driven primarily by architectural inductive biases, pretraining strategy, and domain relevance.
Dina A. Elkholy, Mohamed S. Shehata, John W. Braun· Journal of imaging informati...· 0 citations
A comparative study of different deep learning architectures, including classical CNNs, deep hierarchical models, residual and dense networks, and compound-scaled architectures is presented, showing that deeper networks provide better representation, while residual connections and compound scaling improve training stability and efficiency.
Riyaz Mohammed· International Journal of App...· 0 citations
Vision Transformers (ViT) capture global image context through self-attention but are data-hungry, typically underperforming Convolutional Neural Networks (CNNs) on the small datasets common in fine-grained tasks such as batik motif recognition. This study investigates whether a ViT, trained via knowledge distillation using the Data-efficient Image Transformer (DeiT), can overcome this limitation and compete with CNNs on a small Semarang Batik dataset. A distilled DeiT-Tiny student learns from a ResNet-50 CNN teacher and is benchmarked against two CNN references: ResNet-50 itself (a substantially larger model) and EfficientNet-B0 (a parameter-matched counterpart). In establishing this comparison, we first uncover a critical dataset integrity issue: the publicly available Semarang Batik Dataset (3,020 images) originates from only 18 unique source photographs, each augmented approximately 167 times prior to publication. This near-duplication causes severe data leakage under conventional random splitting, inflating the test accuracy of all models to a misleading 100% and rendering such evaluation meaningless. We therefore introduce a source-aware splitting strategy that enforces group-level separation between training, test partitions, and evaluate all models across three random seeds for statistical reliability. Under this corrected protocol, the distilled DeiT-Tiny attains the highest mean accuracy (95.18 ± 0.30%) and the lowest variance among the three models, matching both the larger ResNet-50 (94.87%) and the parameter-matched EfficientNet-B0 (94.68%) while using only 5.5M parameters. These results confirm knowledge distillation enables a compact Vision Transformer to compete CNNs on a limited fine-grained dataset, and underscore that verifying sample independence is a prerequisite for trustworthy evaluation on pre-augmented public datasets.
Rafi Alifa Bagja, B. Purnama· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
