A Quantitative Explainability Framework for Multimodal Tuberculosis Screening Using Late Fusion and Quadrant-Based Grad-CAM

Abstract

Tuberculosis (TB) remains a major global health challenge, particularly in resource-constrained settings where timely access to expert radiological interpretation is limited. Although artificial intelligence (AI)-based chest X-ray (CXR) screening systems have demonstrated promising diagnostic performance, limited explainability and uncertainty regarding model reliability continue to constrain their clinical adoption. This study presents the design and empirical evaluation of a multimodal decision-level late fusion framework for TB screening that integrates CXR imaging with clinical metadata while introducing a quantitative, quadrant-based Grad-CAM approach for evaluating model explainability. A quantitative, post-positivist experimental methodology was employed using publicly available CXR datasets and associated clinical metadata. A convolutional neural network (CNN) generated imaging-based TB probabilities, while a Random Forest model produced predictions from clinical metadata. The outputs were integrated through decision-level late fusion to generate multimodal TB risk predictions. Explainability was evaluated using Grad-CAM, with the CXR divided into anatomical quadrants to quantify localisation patterns, dominant regions and severity. Perturbation-based analysis was further applied to assess explanation stability and fidelity. Model performance and reliability were evaluated using accuracy, sensitivity, specificity, F1-score, ROC-AUC, Brier score, Expected Calibration Error (ECE), Maximum Calibration Error (MCE), McNemar’s test and decision-curve analysis. The multimodal fusion model achieved an accuracy of 83.75% and ROC-AUC of 0.9137, outperforming the CNN model, which achieved 81.88% accuracy and a ROC-AUC of 0.8696. Fusion also improved specificity, demonstrating its potential to reduce false-positive screening outcomes. Perturbation analysis showed substantial improvements in explanation stability, with prediction variance decreasing by more than 95% for both TB-positive and TB-negative cases. The quadrant-based Grad-CAM framework provided a structured quantitative mechanism for analysing the spatial behaviour and robustness of model explanations rather than relying solely on qualitative visual inspection. These findings demonstrate that combining multimodal prediction with quantitative explainability assessment can improve both diagnostic discrimination and the evaluation of explanation reliability. The proposed framework contributes towards the development of more transparent, measurable and clinically trustworthy AI-assisted TB screening systems, particularly for deployment in resource-constrained healthcare environments.