An Attention-Driven DenseNet–Vision Transformer (ViT) Architecture for Multi-label Chest X-ray Classification

Abstract

Among the most challenging issues with respect to multilabel disease classification through the use of chest X-rays is the difficulty posed by the pattern of lesions that arise in the image besides the presence of class similarities and multiple thoracic diseases appearing in one image. As a result, abnormalities can be hard to see because of the image size in the overall size of the X-ray image and because of the complexity of the relationships between the different disease labels. Therefore, making a correct diagnosis of the diseases using current deep neural network models is very challenging. For these reasons, a new framework is introduced to model both local and global contextual information and also to identify the relationship between the different diseases by developing models. This paper proposes a new hybrid framework by combining both convolutional neural networks (CNNs) and vision transformers (ViT) to overcome the challenges that occur in terms of accurately diagnosing diseases in chest X-ray imaging. The proposed framework utilizes DenseNet121 for extracting local spatial features and ViT for capturing global contextual dependencies from chest X-ray images. Further, the proposed framework incorporates several methods for multi-scale feature learning and feature fusion from the different scales to detect lesions more effectively. Graph-based label dependency learning and contrastive feature learning are two effective approaches that are used to explore the discriminability of features and learn how labels correlate to one another. The hybrid model is tested on the NIH ChestX-ray14 dataset, which has 14 separate categories of diseases. Performance is measured using the evaluation metric Area Under the Curve (AUC). The final hybrid model demonstrates an average AUC score of 0.8303 compared to the baseline DenseNet121 model's AUC score of 0.797, indicating that this hybrid approach is effective in classifying diseases. Furthermore, the hybrid approach demonstrates higher accuracy and reliability than traditional methods.