MMAP is presented, a novel Multimodal Missing-aware Alignment Pretraining method for learning image-tabular representations from incomplete data that outperforms strong multimodal and unimodal baselines on two challenging longitudinal clinical tasks for Alzheimer's disease.
Abstract
Clinical decision making heavily relies on predicting the disease progression trajectory by seeking to understand patient's health status which is characterised by multimodal medical data. AI holds great potential for learning useful representations from multimodal medical data to predict disease progression and aid clinical decision making. However, development of predictive AI models is constrained by missing modalities and incomplete tabular data frequently occurring in medical datasets. In addition, disease labels alone may only provide limited supervisory signals for learning representations from high-dimensional multimodal data. Here, we present MMAP, a novel Multimodal Missing-aware Alignment Pretraining method for learning image-tabular representations from incomplete data. An image encoder is pretrained with efficient sigmoid contrastive learning combined with generative reconstruction. A tabular encoder is built upon a tabular foundation model. A missing token generator enables the two encoders to take incomplete data as input, enabling the model to be robust against missing modalities, either with missing images or missing tabular data. We evaluate the clinical usefulness of the learnt multimodal representations on two challenging longitudinal clinical tasks for Alzheimer's disease: predicting disease stage conversion and predicting amyloid status. The proposed method outperforms strong multimodal and unimodal baselines.
MMOIR-VLM combines missing-modality-aware multimodal encoding, multi-task clinical prediction, and retrieval-grounded natural-language reasoning within a single modular framework.
Sohail Haresh Gidwani, Tamoghna Chattopadhyay, S. Thomopoulos et al.· Frontiers Comput. Neurosci.· 0 citations
Dementia is a major and growing global health burden, with Alzheimer's disease (AD) accounting for most cases. Timely and accurate diagnosis is central to managing this burden and increasingly depends on integrating complementary clinical and imaging information. Multimodal deep learning can combine these modalities fo...
Yusuf Brima, M. Atemkeng, L. Namamula et al.· arXiv.org· 0 citations
Identifying progressive dementia during its prodromal stage is a challenging yet critical task for healthcare providers. Traditional clinical assessments are often not sensitive enough to detect warning signs of cognitive decline. Machine learning (ML) methods have shown promise in modeling clinical trajectories associ...
A. Gautier, L. Zheng, B. Schreurs· medRxiv· 0 citations
Electronic Health Record (EHR) contains varied and diversified clinical data, including demographic data, laboratory measures, diagnosis histories, medication data and physiological observations. However, there remains the issue of effectively using such multimodal information for illness prediction because of the data...
P. B, Raveendrababu Vempati· 2026 International Conferenc...· 0 citations
An imputation approach originally validated under MCAR is extended to instead operate under the more realistic Missing At Random (MAR) mechanism, which proves robust under MAR conditions, with particularly strong performance in cases where missingness is driven by observed variables a property that points toward practi...
Lakshmiprasannakumar Vemavarapu, Chandra Sekhar Sanaboina· International Journal of Com...· 0 citations
This work proposes a multimodal deep fusion framework with attention for high accurate cardiovascular risk stratification using the integration of medical images and clinical data and demonstrates that this adaptive fusion strategy outperforms simple concatenation baselines.
Amit Thakur, Sarita Kumari, Sarita Thakur et al.· Journal of Machine Learning...· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 28, 2026
Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Microsoft Research Blog· microsoft.comAug 11, 2026
Radiology AI is evolving beyond report generation. CARE-X explores a unified approach that combines flexible reasoning, calibrated predictions, and measurement-based tools for chest X-ray interpretation. The post Introducing CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement appeared first on Microsoft Research.