2026· Proceedings of machine learning research· Vol 297, pp. 864 - 885· 1 citation
Medicine
TL;DR
It is shown that the learned model weights generalize effectively to MIMIC-IV radiology reports and that the approach achieves near-optimal performance with only a small amount of labeled training data, which makes the approach practical and reproducible for new settings.
Abstract
Radiology reports contain essential clinical information but often remain in unstructured, free-text formats. Notably, multiple imaging examinations performed simultaneously (such as CT head, facial bones, and cervical spine in trauma cases) may be bundled into a single report that consolidates findings from all studies into one free-text document, written jointly. Because individual sentences may reference ambiguous or overlapping anatomy (e.g., “there is a fracture”), sentence-level anatomic classification—filtering a report to retain only findings relevant to a specific anatomical region—is essential for downstream tasks such as structured label extraction and for creating clean, bijective training data for radiology report generation models. While formatting differs across reports, the clinical language remains precise. Using that fact, we develop context-aware classical models with feature engineering that surpass trained neural networks and pre-trained language models. We show that the learned model weights generalize effectively to MIMIC-IV radiology reports and that our approach achieves near-optimal performance with only a small amount of labeled training data. Together, these results make our approach practical and reproducible for new settings.
Radiology report generation (RRG) has recently benefited from large language models, which substantially improve report fluency. However, clinically faithful generation remains challenging because current supervision is still imposed mostly at the report level. This creates a granularity mismatch: radiology reports are composed of disease-grounded findings, while existing methods are trained mainly with whole-report objectives. To address this problem, we propose Graph-Supervised Hierarchical Clinical Alignment, which reformulates image-report supervision as a hierarchical clinical alignment problem. Our method structures this alignment as a disease-conditioned process, where supervision is decomposed into two levels: Disease-Centric Alignment for fine-grained disease-specific correspondence, and Global Clinical Semantic Alignment for report-level semantic coherence. A clinical knowledge graph is used as a training-time-only structural prior that defines disease-specific supervision units and their clinical relationships, introducing no additional overhead at inference. Because standard contrastive alignment could produce false negatives when studies share overlapping pathologies, we combine instance-conditioned discriminative matching with disease-conditioned soft regularization, enabling fine-grained yet clinically consistent cross-modal representations. Experiments on MIMIC-CXR, IU-Xray, and COV-CTR show that our method consistently improves performance on both conventional and clinical metrics. Notably, our 3B model surpasses several prior systems with larger 7B/13B backbones, suggesting that improving supervision structure, rather than increasing model size, can be more effective for RRG.
Yingshu Li, Yunyi Liu, Zhanyu Wang et al.· 0 citations
Radiology report generation has matured almost entirely on 2D chest radiographs, where the default route to better reports is a larger backbone or a pre-training one on medical data. We revisit that assumption on 3D multi-sequence brain MRI, a volumetric multi-disease regime, and find that the model is not the lever. Zero-shot medical and radiology vision-language models transfer poorly to brain MRI, with chest radiograph specialists failing most conspicuously, and five backbones fine-tuned identically across three model families and an order of magnitude in scale differ only marginally. What determines the quality of the report is the information injected into the prompt. We delegate perception to upstream 3D segmentation and classification, serialize their outputs into a structured fact sentence, and prompt a LoRA-adapted vision-language model with it; we call this \textbf{PerFact}. In a controlled study that fixes the backbone, data split, target reports, and adaptation while varying only the injected grounding, perception-derived facts outperform retrieved prior reports, retrieval becomes redundant once facts are present, and end-to-end predicted facts remain effective without any ground-truth annotation at inference. The residual gap between predicted and oracle facts is explained by the granularity of the facts rather than by the generator. Closed-ended visual question answering comes at no measurable cost to report quality, though the grounding source has little effect on it. On 3D brain MRI, grounding information, not model choice, is the dominant controllable factor in report quality.
Jianyu Sun, Zhenxuan Zhang, Guang Yang et al.· 0 citations
Structured reporting converts free-text radiology narratives into queryable data keys, facilitating cohort assembly, longitudinal tracking, and training label generation for medical AI. The prevailing paradigm follows a two-stage pipeline: (1) constructing a reporting template, (2) extracting information to populate it. While the extraction stage has benefited from advances in large language models (LLMs), template construction remains a manual bottleneck relying on labor-intensive expert consensus that is static, difficult to scale, and may fail to capture real-world reporting diversity. We address this limitation with ASTAR, an LLM-based framework for Automated induction of STAndardized radiology Reporting templates from large-scale clinical free-text corpora. Extensive experiments on 4,215 fetal brain MRI reports from multiple centers demonstrate that, in this reporting scenario, the ASTAR-induced template surpasses two expert-curated templates across template coverage, information fidelity, diagnostic fidelity, and expert-rated usability, reducing template development from weeks of committee deliberation to hours of automated processing.
Breast imaging reports are typically stored as unstructured free-text documents, which limits their use in clinical analytics, research, and downstream computational applications. These challenges are particularly pronounced in Turkish because of its agglutinative linguistic structure, orthographic variability, and the limited availability of standardized clinical corpora. This study presents a data curation framework for transforming heterogeneous real-world Turkish breast imaging reports into structured and machine-readable datasets, with a specific focus on the extraction and standardization of BI-RADS labels already recorded in routine clinical reports. The proposed pipeline integrates domain-specific preprocessing, text normalization, hybrid conclusion-section segmentation, multi-phase BI-RADS label extraction, duplicate and near-duplicate report handling, and modality-based separation within a transparent rule-guided workflow. The framework was applied to two real-world breast imaging datasets comprising 35,104 reports in Dataset 1 and 27,215 reports in Dataset 2 after overlap and duplicate control around the May 2023 transition period. The datasets included ultrasonography, mammography, and magnetic resonance imaging records. The framework achieved conclusion-section segmentation coverage rates of 96.71% and 99.94%, respectively, and BI-RADS label extraction coverage rates of 95.04% and 100.00%, respectively. These coverage values indicate the proportion of reports for which the rule-guided pipeline produced extractable outputs and should not be interpreted as precision, recall, F1-score, or accuracy against an independent gold-standard corpus. The curation process reduced report-level textual redundancy, improved structural consistency, and enabled systematic separation of report narratives from diagnostic assessment labels. Manual review of selected subsets was used as an internal plausibility check rather than as a substitute for independent radiologist-annotated gold-standard validation. By addressing the challenges of structuring free-text breast imaging reports in a low-resource language setting, this study provides a transparent and adaptable methodological basis for future clinical NLP, machine learning, and real-world healthcare analytics.
S. Yildirim, Erkan Ülker, Necdet Poyraz· Applied Sciences· 0 citations
Medical vision-language pretraining (VLP) from paired CT images and radiology reports enables scalable representation learning, but most existing methods align either whole scans with entire reports or local image regions with text fragments. These formulations underuse a key property of radiology reports: findings are organized around anatomical structures, with abnormalities described by organs, disease concepts, locations, and severity-related attributes. We propose OKA-CT, an organ-hierarchical knowledge-augmented framework for CT-report VLP. OKA-CT first converts free-text reports into organ-conditioned knowledge using radiology report parsing and LLM-assisted semantic structuring. The extracted hierarchy is used across two learning stages. Stage~1 injects anatomy-grounded evidence into the CT visual representation through fine-grained organ-conditioned supervision, while Stage~2 uses organ-specific report evidence to guide structured report-CT contrastive learning, where hierarchy-derived semantic soft targets treat non-paired cases with shared organ-level findings as weak semantic positives rather than uniform negatives. A lightweight query-based global branch further aggregates disease-relevant volumetric evidence for whole-scan representation. On CT-RATE and RAD-ChestCT datasets, OKA-CT achieves zero-shot abnormality diagnosis AUROCs of 84.9 and 72.2, outperforming prior CT VLP baselines. Retrieval and patch-occlusion analyses further show improved report-image alignment and stronger sensitivity to disease-associated anatomical regions.
In order to produce meaningful textual interpretations of intricate clinical images, medical image captioning has become a significant field of study at the nexus of computer vision and natural language processing. Due to the lack of explicit modeling of medical entities, current methods frequently fail to produce descriptions that are both semantically valid and clinically useful, despite notable advances in deep learning and vision language modeling. Typical captioning methods in particular, fall short of being able to retrieve fine grained diagnostic information and maintain semantic consistency with clinical findings as they focus on global features. This paper addresses these limitations by presenting an entity-aware medical image captioning approach which aims to identify and incorporate clinically relevant entities into the caption generation, including but not limited to, diagnostic finding, anatomical structures, or diagnostic characteristics. The proposed method utilizes entity level representations as a means of guiding the captioning process thereby ensuring a tighter semantic consistency between visual modalities and the resultant textual output. Consequently, this leads to more comprehensible, informative and clinically relevant generated reports. Additionally, inclusion of entity awareness can aid the model in effectively understanding the relationships between medical concepts leading to captions more consistent with medical expertise. The results demonstrate that explicit modeling with structured semantic information within vision-language frameworks are crucial and that entity-aware methods have the potential to greatly improve captioning. This work has the ability to help advance health intelligence applications which will serve to better assist clinical decision making, scale the processing of medical images, and facilitate accurate medical documentation.
Shaik Rafi, Syed Rizwana, P. Drutika et al.· IEEE Access· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
