A multimodal evidence-driven framework for clinical decision support in cognitive impairment.

An interpretable multimodal ensemble assessment framework for Alzheimer’s disease cognitive staging

Objectives Alzheimer’s disease (AD) is the most prevalent type of neuro-degenerative dementia. Artificial intelligence assisted clinical evaluation can improve diagnostic efficiency and facilitate timely intervention. Methods An interpretable Multimodal Ensemble Assessment Framework (MEAF) was developed to support clinical evaluation for AD across the cognitive spectrum. This framework employed a Swin Transformer and ResNet-50 for staged modeling of imaging features, applied machine learning techniques to extract clinical features, and used decision-level ensemble learning to integrate multimodal predictions. For interpretability analysis, Gradient-weighted Class Activation Maps were used to highlight key brain regions contributing to imaging-based decisions, and Shapley Additive exPlanations were applied to quantitatively assess the importance of clinical features. Results MEAF achieved robust performance in classifying cognitively normal, mild cognitive impairment, and AD, with an accuracy of 0.878 and an F1-score of 0.877. In the independent external validation cohort, MEAF maintained reasonable performance, with an accuracy of 0.817 and an F1-score of 0.803. Interpretability analyses provided complementary explanations for both the imaging and clinical models. Conclusion MEAF demonstrated favorable classification performance and interpretability in retrospective multicenter datasets, suggesting its potential as an auxiliary framework for multimodal assessment of AD-related cognitive staging.

An explainable AI framework integrating deep learning and large language model for student’s mental health

Mental health disorders such as anxiety, depression, and Mild Cognitive Impairment (MCI) are increasingly prevalent among young adults aged 18–30, significantly affecting academic performance, cognitive functioning, and overall wellbeing. Traditional diagnostic approaches depend on subjective assessments and limited clinical observations, making early and accurate detection challenging. To address these limitations, this research proposes an interpretable deep learning-based multimodal method for comprehensive mental health prediction and personalized intervention. The framework integrates heterogeneous data sources, including demographic, cognitive, behavioral, physiological, and neurocognitive indicators collected from clinical settings. Data preprocessing includes imputation, normalization, encoding, and text transformation. A Cross-Directional Feature Learning Network (CDFLN) is employed for robust multimodal feature extraction, followed by a Multi-model Progressive Dense Self-Attention for Cross Domain (MPDSA-CD) architecture for classification of anxiety, depression, and MCI, along with cognitive risk and severity assessment. Model performance is further enhanced by the Starfish Optimization Algorithm for hyperparameter tuning and parameter refinement. To ensure clinical transparency, SHapley Additive exPlanations (SHAP) are utilized to interpret model predictions and identify key risk factors influencing mental health outcomes. The proposed method achieves an accuracy of 99.8%, precision of 99.7%, recall of 99.9%, and F1-score of 99.8%, demonstrating strong robustness, generalization ability, and clinical applicability for early detection and effective psychological intervention in young adults.

Interpretable multimodal learning for integrating neuroimaging and genetic data in Alzheimer’s disease

Introduction Early detection of Alzheimer's disease (AD) requires models that combine brain structure changes with genetic risk, but existing methods struggle to align these different data types. Methods We present R-GenIMA, an interpretable multimodal large language model that pairs a region-of-interest vision transformer with genetic prompting to jointly analyze structural MRI and single nucleotide polymorphisms (SNPs). Each brain region becomes a visual token and SNP profiles are encoded as structured text, letting the model link regional atrophy to genetic factors through cross-modal attention. Tested on the ADNI cohort, R-GenIMA performs well in classifying four groups: normal cognition, subjective memory concerns, mild cognitive impairment, and AD. Results Beyond accuracy, it produces biologically meaningful explanations, identifying stage-specific brain regions and genes. The model consistently highlighted known AD risk genes (APOE, BIN1, CLU, RBFOX1) and revealed stage-specific patterns: striatal involvement in subjective decline, frontotemporal changes in early impairment, and broad network disruption in AD. Discussion These results show that interpretable multimodal AI can integrate imaging and genetics to reveal disease mechanisms, providing a foundation for clinical tools that enable earlier risk assessment and inform precision treatment in Alzheimer's disease.

Data-centric, robust, and explainable multimodal deep learning for clinical decision support: A systematic review.

PURPOSE Multimodal deep learning is increasingly proposed for clinical decision support (CDS) under a "data-centric" framing that prioritizes label quality, missing-modality robustness, distribution shift, calibration, and explainability. Prior reviews have examined multimodal medical AI, CDS, and data-centric methods separately, but none address their intersection. We mapped the modalities, fusion strategies, and data-centric and explainability techniques used in this recent literature, quantified how often each is implemented rather than merely mentioned, assessed deployment-relevant evidence (external validation, clinical-outcome measurement, equity), and formally appraised study-level risk of bias. METHODS Following the PRISMA 2020 statement (PROSPERO CRD420261427815; registered retrospectively), we screened 150 records and included primary, clinical, multimodal studies that applied machine or deep learning to a decision-support task and reported at least one quantitative result. Two reviewers screened and extracted data with consensus adjudication. Each study was coded against pre-specified operational definitions, separating implemented or empirically evaluated techniques from those only mentioned. Study-level risk of bias was assessed with PROBAST + AI. Synthesis was narrative. RESULTS Thirty-one studies met inclusion; 30 (97%) were published between 2024 and 2026, with a median of three modalities (range 2-6), most commonly structured EHR (71%) and imaging (39%). Data-centric techniques were frequently reported (74-84% across label-noise, distribution-shift, calibration, missing-modality and class-imbalance handling; equity 61%). However, external validation was reported in only 4/31 studies (13%), a clinical or provider outcome in 3/31 (10%), and no study reported routine deployment. Overall risk of bias was high in 27/31 studies (87%), driven by the analysis domain. CONCLUSION Within this recent, self-selected slice of the field, technical robustness and explainability techniques are widely reported but rarely validated out-of-distribution or against clinical outcomes, and the underlying evidence is at high risk of bias. Progress requires external multi-site validation, clinical-outcome measurement, formal bias appraisal, and adherence to AI reporting standards (e.g., TRIPOD + AI) before deployment can be justified.

A Vision-Language Framework for Predicting Brain Tumor Recurrence from Multimodal, Longitudinal Patient Data

Accurate prediction of tumor recurrence in brain tumor patients following surgery is essential for optimizing adjuvant therapy, response assessment, and surveillance regimen. While MRI remains the gold standard for surveillance, integrating patient-specific clinical context may inform recurrence prediction. Traditional multimodal deep learning approaches often incorporate clinical data via simple fusion, failing to fully capture the semantic interdependencies between visual features and clinical context. Trained on over 5,000 scans from approximately 400 pediatric low-grade glioma subjects and validated across three institutional cohorts, including one clinical trial cohort, our experiments demonstrate incremental performance gains when progressing from vision-only to clinical-vision to a vision-language approach. Our results indicate that converting structured clinical covariates into natural language text allows for more effective synthesis of multimodal data, while providing a platform for incremental addition of clinical context without extending model complexity. We demonstrate that our proposed VLM architecture offers a promising direction for neuro-oncological prognosis by effectively encoding imaging cues and clinical context, with potential applicability to other longitudinal prognosis tasks.

Explainable Machine Learning Models for Alzheimer's Diagnosis Using Routine and Low-Cost Clinical Data