MedCure: Medical Data Curation for Efficient Vision–Language Pretraining

Abstract

Medical foundation models (FMs) typically rely on contrastive vision-language pretraining over large-scale datasets, yet such datasets often exhibit substantial heterogeneity in data quality and demand extensive computational resources. Recent studies suggest that data quality matters more than data quantity, but how to identify a compact subset of pretraining data that best supports generalization remains unclear. In this paper, we propose an active Medical data Curation framework for efficient vision-language pretraining (named MedCure), which jointly prioritizes two key properties of training data: difficulty and diversity. Specifically, during pretraining, MedCure establishes undirected data graphs to characterize sample importance, dynamically updating the difficulty score of each sample by transferring and aggregating information from its neighbors within the graph. These refined scores are then leveraged by a backward difficulty transfer mechanism to adaptively curate subsets that comprehensively capture both difficult and heterogeneous regions of the data space. Extensive experiments on a chest X-ray database of over 1.2M image-text pairs illustrate that MedCure achieves comparable results to full-data training using only 25% of the pretraining data and 32.7% of the full training cost, as evaluated on a variety of zero-shot generalization and downstream-adapted tasks (e.g., disease classification, cross-modal retrieval, lesion and anatomy segmentation, and radiology report generation), highlighting its effectiveness for data-efficient medical FMs. Our code and pretrained models will be available at https://github.com/sendyma/MedCure