This work proposes MedVCoT, which incorporates latent visual reasoning into the medical visual question answering (VQA) domain, and utilizes the specialized expertise of MedSAM to train a large vision-language model so that it can autonomously generate consistent and continuous latent visual tokens within Visual Chain-of-Thought.
Experiments on OmniMedVQA subsets show that Med-OPD consistently outperforms SFT and standard OPD across CT, MRI, Disease Diagnosis, and Lesion Grading, and demonstrate that evidence-aware distillation can better strengthen medical VLMs'reliance on key visual evidence and improve reliable multimodal medical reasoning.
Yun-Hang Qian, Jiaquan Yu, Jiawei Liu et al.· arXiv.org· 1 citation
A novel multimodal RAG framework tailored for MedVQA is proposed, which leverages multimodal data, including medical images, reports, and generated captions, to provide more accurate clinical answers, and introduces a training paradigm that uses captions as auxiliary supervision, enhancing cross-modal alignment via contrastive learning.
Mai A. Shaaban, M. Zarei, Adnan Khan et al.· 0 citations
A benchmark score credits final answers, but not the route by which an item can be answered. In medical multimodal multiple-choice questions (MCQs), this distinction matters because a correct answer can be supported by the intended image finding or by benchmark-preserved cues in the wording of answers, non-visual clinical text, visible image text, artificial annotations, or device/context artifacts. We call the resulting score-level overinterpretation reasoning inflation. Here, a route is an observable input path that can support answer selection, not a claim about the model's hidden cognition. Across six medical multimodal MCQ datasets, we separate candidate cues from behavioral evidence through prompt- and image-side audits, modality ablations, and matched repairs that preserve the medical target and answer key. In a 13-configuration open-model panel, full-input accuracy is 62.63%, while text-only and options-only settings achieve 53.96% and 29.71%, respectively. Removing length-gap, absolute/conspicuous, and spatial/prepositional cues lowers accuracy by 6.58, 3.50, and 4.77 percentage points. We also construct MedQA-MM, a 1,000-item shortcut-mitigated subset, where text-only and options-only accuracy fall to 5.21% and 12.33%. This does not imply that models never use images; it shows that medical image-reasoning claims require route-level evidence.
Ben-Lu Wang, Yi-Fan Zhang, Jia-Qing Yu et al.· 0 citations
Multimodal Large Language Models (MLLMs) excel at understanding generic visual content, such as landscapes, objects, and events, thanks to extensive datasets and advanced training regimes. However, their effectiveness in medical applications remains limited due to the inherent discrepancies between data and tasks in medical scenarios and those in the general domain. Existing medical MLLMs face the following critical deficiencies: 1) inadequate coverage of medical knowledge beyond imaging; 2) elevated propensity for hallucinations due to suboptimal data curation; and 3) limited reasoning capacity tailored to complex medical tasks. To address these challenges, we first propose a comprehensive data-curation procedure that 1) efficiently acquires rich medical knowledge data not only from medical imaging but also from extensive medical texts and general domain data; and 2) synthesizes high-quality medical captions, visual question answering, and reasoning samples. Leveraging the curated data, we build a multimodal dataset imbued with extensive medical knowledge and develop our medical-specialized MLLM, Lingshu-Med, which undergoes multi-stage training to embed the medical expertise and enhance task-solving capabilities progressively. We also investigate reinforcement learning with verifiable rewards to further refine Lingshu-Med's medical reasoning abilities. For rigorous assessment, we introduce MedEvalKit, a unified evaluation framework that consolidates the leading multimodal and textual medical benchmarks for standardized, fair, and efficient model assessment. On three core medical tasks-multimodal QA, textual QA, and radiology report generation, Lingshu-Med consistently outperforms existing multimodal baselines in most tasks. Moreover, we conduct five case studies drawn from real-world clinical scenarios that illustrate its practical utility in medical contexts.
Wei-Wen Xu, H. Chan, Long Li et al.· IEEE Transactions on Pattern...· 0 citations
A definitive taxonomy of the medical VLM landscape is provided, tracing the evolution from early Contrastive Alignment and Generative MLLMs to the cutting-edge frontiers of Dense Pixel-Grounding, Sparse Mixture-of-Experts (MoE), and Reasoning-Incentivized (RL) architectures.
Taha Razzaq, Murtaza Taj, Asim Iqbal· Journal of Biomedical Inform...· 0 citations
LocAnyMed-CoT-20K is derived, a rationale-augmented subset that connects anatomical context, visual observations, and spatial conclusions through structured reasoning and further improves cross-source generalization through fine-tuning.
Zi-hao Wang, Tong Liu, Zhiwei Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
