Jul 2026· DMPedia Lecture Notes in Computer Science & Engineering· pp. 438-450· 0 citations· 19 references
TL;DR
A Multimodal Brain Tumour Classification and Segmentation framework based on a Dual-Attention Swin-UNet architecture, enhanced with a Quantum-Inspired Optimisation (QIO) technique, to overcome limitations in transformer-based medical imaging models.
Abstract
One of the most significant and challenging challenges in medical image processing is the use of magnetic resonance imaging (MRI) to diagnose brain cancers. Significant tumour heterogeneity, irregular morphologies, blurred boundaries, and differences between imaging modalities are the root causes of the problem. Early and precise disease detection is critical for better treatment planning and patient outcomes. Automated brain tumour analysis has greatly improved because of deep learning, particularly convolutional neural networks (CNNs). However, with high-resolution medical images, traditional CNN-based models frequently struggle to identify long-range contextual linkages. This limitation may make it more difficult to obtain precise segmentation and reduce the overall reliability of the diagnosis. Transformer-based designs have recently shown great potential in visualising global relationships. However, because they are hard to train, require a lot of memory, and are rather complex to utilise, their practical application in medical imaging is challenging. We provide a Multimodal Brain Tumour Classification and Segmentation framework based on a Dual-Attention Swin-UNet architecture, enhanced with a Quantum-Inspired Optimisation (QIO) technique, to overcome these limitations. The proposed system successfully integrates fine-grained spatial localisation and global contextual awareness by combining a U-Net-style decoder with an encoder based on Swin Transformer. Dual attention mechanisms spatial and channel attention are added to enhance feature representation and facilitate visualisation of tumour edges, thereby further accelerating feature learning. These attention modules filter away irrelevant background noise, allowing the network to concentrate on clinically significant areas. Additionally, we introduce a quantum-inspired optimisation+ technique to reduce training oscillations and enhance convergence stability, which are frequently observed in transformer-based medical imaging models. A pre-processed version of the BraTS 2020 dataset and additional brain MRI images recorded in HDF5 format are used to train and evaluate the framework. This enables more effective data handling and large-scale training. Due to computational power constraints, GPU acceleration is used for full-scale model training. A real-time, user-friendly web-based diagnostic interface demonstrates a lightweight and deployable inference pipeline. The experimental outcomes demonstrate the strength and clinical practicability of the proposed method, which offers superior tumour segmentation accuracy and reliable classification. The possible development of scalable and intelligent clinical decision support systems for brain tumour diagnosis through the integration of transformer-based architectures, the geography of attention processes, and quantum-inspired optimisation strategies is identified in the study.
A hybrid deep learning framework consisting of a U-Net architecture integrated with Long Short-Term Memory (LSTM) networks is designed, which displays the promise of combining a convolutional and a recurrent architecture to propagate automated neuroimaging analysis.
K. Pranitha, Vuda Sreenivasa Rao· Pertanika journal of science...· 0 citations
Early and accurate discovery of brain tumours is crucial for timely diagnosis and effective treatment planning. In this work, we put forward an explainable multimodal CNN–Transformer framework, kind of merges MRI and CT cues in a complementary way, using attention-guided multimodal fusion so that tumour classification and localisation can happen at the same time. Instead of most existing multimodal CNN–Transformer methods, which mostly linger on feature merging for classification only, this proposal does a bit more. It uses modality-specific CNN encoders, a Transformer based cross-modal attention module, and several fusion recipes—early fusion, late fusion, and attention-driven fusion. On top of that we do quantitative Grad-CAM checking against expert radiologist annotations, plus a computational efficiency examination, all inside a single coherent architecture. Overall this should boost interpretability and make the approach more clinically usable. For training and evaluation, the framework was tested with a carefully curated dataset of 1,986 patient-level MRI–CT pairs collected from BraTS 2021 and the TCIA Brain Tumour CT Archive. The dataset includes glioma, meningioma, pituitary tumour, and healthy controls. We used a patient-wise split: 70% for training, 15% for validation, and 15% for testing. When we looked at the attention based fusion approach, it delivered classification accuracy of 96.2 ± 0.3% and an AUC of 0.98. For localisation, Grad-CAM alignment produced a Dice overlap of 0.84 ± 0.04 versus annotations from three radiologists, across five separate experimental runs (reported as mean ± SD). Beyond the metrics, the framework also looks practical for real-world deployment. Inference time was 0.18 s per image, computational complexity reached 9.3 GFLOPs, and GPU memory use was 5.6 GB. So, these findings suggest that the proposed framework delivers an accurate, understandable, and compute friendly solution for multimodal brain tumour diagnosis, with clear benefits for clinical decision support.
Diptee Ghusse, Harshala Shingne, A. Yenkikar et al.· Discover Computing· 0 citations
The problem of accurate identification of brain tumors using multi-modal MRIs still poses significant challenges due to tumor heterogeneity and variations across different image modes, as well as inconsistent availability of imaging modalities in real-world applications. Although deep learning algorithms such as convolution and transformer networks have demonstrated high efficacy in tumor segmentation through effective modeling of local and global contextual information, many state-of-the-art models perform directly on concatenated or fused images, making it difficult to leverage differences in information provided by different modalities and leading to inferior performance in cases of modality imbalance or missing data. To tackle the issue, we present in this paper a novel multimodal transformer network using the concept of reliability-driven modality attention for robust brain tumor segmentation. Our approach employs a feature extraction pipeline with a reliability estimator that automatically calculates weighting coefficients for each input modality (T1, T2, FLAIR, T1-CE), enabling more efficient feature representation than traditional fusion techniquesFurthermore, a slice-aware 2.5D context modeling strategy is used to capture inter-slice dependencies while keeping computational efficiency high compared to full 3D models. Extensive experiments on benchmark multi-modal MRI datasets show that the proposed approach achieves better segmentation performance than leading CNN, transformer, and hybrid methods, especially in scenarios with missing or degraded modalities. The results emphasize how reliability-aware fusion improves robustness, generalization, and clinical use of automated brain tumor analysis systems.
C. Nirmala, T. R. Ganesh Babu· International Conference on...· 0 citations
Brain tumor segmentation from Magnetic Resonance Imaging (MRI) is an important task in computer-aided diagnosis because accurate identification of tumor regions supports clinical assessment and treatment planning. However, the complex structure, irregular shape, intensity variation, and heterogeneous appearance of brain tumors make automated segmentation challenging. This study presents a comparative deep learning framework for brain tumor segmentation using the BraTS 2020 dataset and two-dimensional (2D) MRI images. In this study, we explore 2D deep learning architectures for automated tumor segmentation, focusing on U-Net, Vision Transformer (ViT), and Res-ViT models. U-Net, with its encoder–decoder design and skip connections, has been widely adopted for medical image segmentation due to its ability to capture fine-grained spatial features. ViT, leveraging self-attention mechanisms, introduces a global receptive field that enhances contextual understanding across slices. The Res-ViT hybrid combines residual learning with transformer-based attention, aiming to balance local feature extraction and long-range dependency modelling. Preprocessing steps, including skull stripping, intensity normalisation, and bias field correction, were applied to ensure consistency across scans. Data augmentation techniques such as rotation, flipping, and elastic deformation were employed to mitigate overfitting and improve generalisation. The models are evaluated using important segmentation metrics, including Intersection over Union (IoU), accuracy, precision, recall/sensitivity, loss, and 95th-percentile Hausdorff Distance (HD95). The comparative analysis aims to identify the strengths and limitations of convolutional and transformer-based approaches for 2D brain tumor segmentation. The study demonstrates the potential of combining local feature extraction and global contextual learning to achieve more accurate and robust brain tumor segmentation from multimodal MRI images.
Keywords: MRI; U-Net; ViT; BraTS 2020; IOU; HD95.
Lovedeep Kaur, Parminder Singh, Naveen Dhillon· International Journal of Com...· 0 citations
Results show that ViT–BiLSTM's classification performance is superior to those of traditional deep learning methods: among all the tumor categories its accuracy is higher, its fine-tuning more perfect, as well as, its Recall rates greater.
Nagham Salim Mohammed, Omar S. Almolaa, A. S. Abdullah et al.· ITEGAM- Journal of Engineeri...· 0 citations
This study proposes BrainTumor CNN, a convolutional neural network (CNN) for classifying brain tumor MRI images, which leverages transfer learning via a pre-trained ResNet-18 network, integrating data augmentation and Dropout regularization to enhance robustness and generalization.
Yicheng Xu· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
