Diagnosis of Alzheimer’s disease based on 3D MRI features and large language model embeddings

Abstract

Alzheimer’s disease (AD) is the most common type of dementia worldwide. As cognitive functions decline, it severely affects the quality of life and health of patients. Early and accurate diagnosis of AD is crucial for timely treatment intervention and improving the prognosis of patients. However, the current clinical methods for diagnosing AD mainlyrely on doctors' subjective clinical assessment and interpretation of neuroimaging data. These methods are time-consuming, labor-intensive, and have variability. This study aims to propose a multimodal deep learning framework that integrates 3D magnetic resonance imaging (MRI) features and clinical text embeddings based on large language model (LLM) to achieve AD classification diagnosis. The proposed multimodal deep learning framework includes a3DResNet-18 backbone network for MRI feature extraction and a pre-trained LLM embedding module (ZhipuAIEmbedding-3) for clinical text representation. This framework was evaluated on the Alzheimer's Disease Neuro imaging Initiative (ADNI) dataset. In the three binary classification tasks, the performance of the proposed multimodal fusion model was significantly better than that of the model based on sMRI images and some existing methods.