This work proposes Co-occurrence Weighted Adaptation (CoWA), which leverages disease co-occurrence patterns as a reliability signal for adaptation, enabling adaptation to rely more on consistent predictions while reducing the impact of noisy ones.
Abstract
Medical imaging models often degrade when deployed at new clinical sites due to differences in imaging equipment, protocols, and patient populations. Test-time adaptation (TTA) addresses this by updating a pretrained model using only unlabeled target data, without access to source data. However, existing TTA methods were designed for single-label classification on natural image benchmarks, minimizing entropy uniformly across all samples without considering label dependencies. This overlooks a key property of multi-label medical imaging: pathologies do not occur independently but exhibit structured co-occurrence patterns. In this work, we propose Co-occurrence Weighted Adaptation (CoWA), which leverages disease co-occurrence patterns as a reliability signal for adaptation. CoWA estimates label co-occurrence structure from model predictions and downweights samples that deviate from expected patterns, enabling adaptation to rely more on consistent predictions while reducing the impact of noisy ones. We evaluate CoWA on chest X-ray benchmarks under domain shifts and demonstrate consistent improvements over established baselines.
Recent vision-language models for chest X-ray understanding are largely built on image-report alignment and therefore rely heavily on MIMIC-CXR as the dominant pretraining source. While effective at scale, this paradigm underexplores an important alternative source of supervision: a range of existing multi-label classification datasets, which provide cleaner and more explicit disease signals than free-text reports, and can offer broader pathology coverage when combined across sources. However, learning from such heterogeneous datasets is nontrivial, as differences in label ontologies, annotation protocols, acquisition pipelines, and report styles can cause models to entangle clinical semantics with dataset identity, leading to poor transfer despite increased scale. In this work, we revisit radiology VLM construction from the perspective of harmonized multi-source learning. We propose HarMoE, a dataset-aware mixture-of-experts framework that learns shared cross-dataset medical semantics while confining source-specific variation to lightweight residual experts in deeper decoder layers. To further exploit clean supervision from labeled datasets, we train in a unified disease vocabulary with masked multi-dataset supervision, enabling the model to leverage complementary annotations without introducing false negatives. Experiments on large-scale chest X-ray benchmarks show that HarMoE consistently improves zero-shot classification, out-of-distribution transfer, and grounding over strong baselines. Our results suggest that building robust radiology VLMs requires moving beyond single-source image-report alignment toward structured knowledge construction from heterogeneous datasets with cleaner supervision and broader coverage. Code and the 873k harmonized dataset will be released at https://github.com/Roypic/harmoe.
Haozhe Luo, Zi-Yu Zhou, Shelley Zixin Shu et al.· 0 citations
Test-time adaptation (TTA) has become a practical way to adapt deployed models to unlabeled target data, a setting that is especially relevant in computational pathology where staining, scanner, and cohort shifts are routine. While most TTA methods are evaluated by their effect on accuracy, clinical use also depends on whether the model's explanations remain reliable after adaptation. In this paper, we take a closer look at this largely unmeasured effect. We study explanation stability under TTA across two histopathology benchmarks, Camelyon17 and NCT CRC-HE, using five architectures ranging from convolutional networks to vision transformers and a pathology foundation model, seventeen TTA methods, and four attribution families. Across 2,958 adaptation runs, we observe a clear and systematic pattern: TTA methods differ sharply in how much they move model explanations, with frozen-backbone methods leaving attributions almost unchanged and continual methods such as CoTTA and RoTTA causing the largest drift. This effect is not uniform. Convolutional networks are substantially more sensitive than transformer and foundation-model backbones, and explanation drift increases with adaptation strength while remaining largely insensitive to batch size. Surprisingly, explanation stability is only weakly coupled to adaptation quality. Some methods preserve explanations almost perfectly while degrading calibration or accuracy, producing silent failures that would be missed by accuracy-only or explanation-only evaluation. These findings show that explanation stability is a distinct reliability axis for TTA in computational pathology. We release the metric, protocol, and full benchmark to support future work on adaptation methods that are not only accurate, but also stable and clinically auditable. Code: https://github.com/bahumanyarg11/tta-explanation-stability-pipeline
R. G. Bahumanya, M. HarshithV., S. Gowda et al.· 0 citations
This work systematically investigates how evaluation-reference choices affect model performance and ranking in both pathology classification and image quality assessment (IQA), and shows that for supervised image classifiers, changing the label source leads to substantial differences not only in performance estimates but also in model rankings.
Panagiotis Fytas, Ian Selby, C. Karner et al.· arXiv.org· 0 citations
This work proposes a label-free selection criterion built on SUDO, a framework for evaluating clinical AI systems without ground-truth annotations, and shows that AURCC can be used to rank a variety of vision-language models on chest X-ray classification across three inter-hospital shift scenarios, under zero-shot and MLP-probe regimes.
Juan Iñaki Larrea, L. Mansilla, Enzo Ferrante· 0 citations
Vision-Language Models (VLMs) for radiology report generation are typically trained on retrospective clinical reports, which suffer from omission noise: clinically present findings are left unreported due to the omission of subtle findings. For example, prior studies show that cardiomegaly may be omitted from ICU chest X-ray reports when the imaging request is focused on monitoring support device placement. As a result, models trained with standard approaches inherit these omissions, learning to under-report findings themselves. We propose PU-DPO, a preference optimization framework to prevent omission noise from corrupting the preference signal. We reformulate the objective under a positive-unlabeled (PU) learning framework, treating absent mentions as unlabeled rather than truly negative. Our framework provides preference supervision using constructed contrastive pairs, generated using edits to model responses, producing variants that explicitly mention or omit a specific finding. Generated responses that mention the finding are naturally preferred in the context of visual evidence. Across semi-synthetic experiments and analyses on real-world chest radiograph benchmarks where adjudicated labels are available, PU-DPO yields consistent gains in detection rates and recovery of hidden positives across multiple pathologies, and is more robust to omission noise than prior approaches.
Y. Kobayashi, P. Ramesh, Muhammad Ahmed Chaudhry et al.· 0 citations
Computed Tomography (CT) scans are widely used to diagnose lung infections; however, manual interpretation is labor-intensive. Artificial intelligence has accelerated the development of computer-aided diagnostic (CAD) systems, allowing faster and more accurate diagnosis. Nevertheless, many existing CAD systems lack robust cross-dataset generalization and interpretability, limiting their reliability and resulting in suboptimal diagnostic performance. To address these limitations, we propose a semantic attention-driven retrieval framework based on a lightweight Meta-Domain Adaptive Segmentation Network (MDA-SN) with an adaptive data normalization strategy to enhance infection detection in cross-dataset analysis. This framework quantifies infection ratios and retrieves relevant CT slices from the database, closely matching the input test sample to further support medical experts in making more accurate diagnostic decisions. The MDA-SN design leverages multi-scale dilated grouped convolution with residual attention to ensure real-time performance while maintaining accuracy. Our framework achieved an average cross-dataset performance of 75.93% Dice index and 67.42% Intersection over Union, surpassing state-of-the-art methods by 3.32% and 3.28%, respectively. Additionally, it achieves real-time execution, processing an average of 29 slices per second, due to its significantly reduced number of training parameters, approximately 70% fewer than its closest competitor. The implementation and materials are available at our GitHub repository:
https://github.com/Owais-CodeHub/MDA-SN
.
Muhammad Owais, Taimur Hassan, Naqash Afzal et al.· Scientific Reports· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
