Aug 2026· Nature Biomedical Engineering· 0 citations· 43 references
Medicine
TL;DR
Across a large-scale benchmark covering 78 datasets in 10 imaging modalities, ConceptCLIP demonstrates superior diagnostic performance while providing human-understandable explanations, and represents a critical milestone towards the widespread clinical adoption of AI.
Abstract
Artificial intelligence for medical imaging is required to be accurate and interpretable to clinicians. However, current multimodal biomedical foundation models often prioritize performance over explainability. Here we present ConceptCLIP, an explainable biomedical foundation model that achieves state-of-the-art diagnostic accuracy while delivering human-interpretable explanations across diverse imaging modalities. We curate MedConcept-23M, a large-scale dataset comprising 23 million biomedical image-text-concept triplets. Leveraging this dataset, we pretrain ConceptCLIP via joint image-text and region-concept alignment for precise and interpretable medical image analysis. Across a large-scale benchmark covering 78 datasets in 10 imaging modalities, ConceptCLIP demonstrates superior diagnostic performance while providing human-understandable explanations. In a clinician user study spanning three modalities, the concept-based explanations provided by ConceptCLIP help clinicians verify model predictions and identify potential errors. As an explainable biomedical foundation model, ConceptCLIP represents a critical milestone towards the widespread clinical adoption of AI, thereby advancing trustworthy AI in medicine.
Large language models (LLMs) have demonstrated strong capabilities across diverse domains, showing considerable potential in medicine. However, their application in medical settings remains limited by the scarcity of visual question answering (VQA) datasets that capture clinical reasoning and explicit image-text alignm...
Ling-Xuan Hou, Yu-Hua Xie, Yue Hu et al.· 0 citations
A FOundational LLM Trained on ThoughtMed-1M (FOLTMed), a scalable paradigm for advancing research on clinically grounded multimodal LLMs, achieved state-of-the-art performance across 42 medical VQA benchmark datasets, with a macro accuracy of 85.4%, and generated more clinically coherent responses on the ThoughtMed-1M...
Ling-Xuan Hou, Yu-Hua Xie, Yue Hu et al.· 0 citations
Experiments on BUS-CoT and IU X-ray datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs, indicating that concept-grounded reasoning better aligns generation with clinical decision processes.
Xin-Yue Xu, Hong-Bin Lin, Juan-Gui Xu et al.· 0 citations
Medical foundation models (FMs) typically rely on contrastive vision-language pretraining over large-scale datasets, yet such datasets often exhibit substantial heterogeneity in data quality and demand extensive computational resources. Recent studies suggest that data quality matters more than data quantity, but how t...
Chong Wang, Feng-Bei Liu, Yu-Yuan Liu et al.· IEEE Transactions on Image P...· 0 citations
HCE-CLIP is proposed, a vision–language pretraining framework that formulates medical image–text alignment as a hierarchical label-conditional energy modeling problem and introduces a hierarchical contradiction-based metric that quantifies logical inconsistencies between fine-grained disease predictions and higher-leve...
Cheng-Sheng Mao, Yuan Luo· Proceedings of the Thirty-Fi...· 0 citations
This work introduces InfiMed2, a family of 4B and 27B generalist medical multimodal foundation models built around stage-aware data design, and curates a 55.68B-token corpus that combines broad clinical knowledge with context-rich biomedical visual evidence through source-specific processing.