Lung Cancer Detection and Malignancy Classification Using Custom CNN and Transfer Learning with ResNet50 and VGG16 Augmented by Explainable Artificial Intelligence

Abstract

Lung cancer is the leading cause of cancer-related mortality worldwide, accounting for approximately 1.8 million deaths annually. Early and accurate discrimination between malignant and non-malignant pulmonary nodules in computed tomography (CT) images is clinically critical, as five-year survival rates improve from under 10% for late-stage detection to over 80% when malignancy is identified at Stage I. This paper proposes a comprehensive deep learning framework for binary classification of lung CT scans into Malignant and Non-Malignant categories. Three convolutional neural network architectures are designed, trained, and benchmarked: a bespoke Custom CNN, and two transfer learning models based on ResNet50 and VGG16 pretrained on ImageNet. Custom CNN achieves an overall accuracy of 87.2%, VGG16 attains 91.8%, and ResNet50 reaches a peak accuracy of 94.1% with a sensitivity of 96.0% the most clinically critical metric for cancer screening. To address the interpretability imperative of AI assisted medical diagnosis, the framework integrates Local Interpretable Model Agnostic Explanations (LIME) and SHapley Additive explanations (SHAP) as complementary post hoc Explainable AI (XAI) modules. LIME generates superpixel saliency maps localising nodule boundaries, while SHAP provides pixel level Shapley value attributions identifying radiological features nodule density, margin spiculation, and ground glass opacity as the primary malignancy indicators. Rigorous evaluation via 2×2 confusion matrices, ROC curves (ResNet50 AUC=0.978), Precision Recall curves, and per class classification reports confirms ResNet50 as the clinically superior model, offering a viable AI assisted diagnostic pathway for radiologists.