A Leakage‐Free, Patient‐Level Evaluation Protocol for Multi‐Class Lung CT Diagnosis

Abstract

Medical imaging artificial intelligence can produce overoptimistic performance estimates due to data leakage, where information from the same patient is used in both the training and test sets. Although awareness of this issue is increasing, slice‐level data partitioning is still reported or insufficiently specified in some studies based on volumetric imaging. This paper offers a systematic patient‐level evaluation protocol that aims to reduce leakage and quantify its effect in an experimental setting. Based on the publicly accessible IQ‐OTH/NCCD lung CT dataset comprising 110 patients, three convolutional neural network models, ResNet50, DenseNet121, and EfficientNetB3, were tested through stratified patient‐level 5‐fold cross‐validation with automatic verification of patient separation. At the patient level, the models achieved accuracies of 81.3%, 82.8%, and 83.1%, with AUC values above 0.87 across all models. Leakage‐prone slice‐wise splitting produced higher apparent performance than patient‐disjoint evaluation, and an additional unit‐matched slice‐level analysis confirmed that this difference was not explained solely by patient‐level aggregation. These results show that evaluation design alone can substantially affect reported diagnostic performance. The suggested framework provides a clear and reproducible protocol aligned with CLAIM and TRIPOD+AI reporting principles, supporting more trustworthy evaluation of medical AI systems developed using multi‐slice imaging data.