A fully on-device AI pipeline using open-source Automatic Speech Recognition and Large Language Models that transcribes breast cancer MDT discussions, structures clinical information, and generates treatment recommendations using retrieval-augmented generation (RAG) grounded in National Institute for Health and Care Excellence (NICE) guidance is developed.
Abstract
Breast Cancer Multidisciplinary Team (MDT) meetings manage increasingly complex cases under considerable time pressure, and documentation requirements can reduce clinical efficiency and decision quality. Existing AI based MDT workflows rely on cloud-based processing, limiting their use because patient discussions contain identifiable information. We developed a fully on-device AI pipeline using open-source Automatic Speech Recognition (ASR) and Large Language Models (LLMs) that transcribes breast cancer MDT discussions, structures clinical information, and generates treatment recommendations using retrieval-augmented generation (RAG) grounded in National Institute for Health and Care Excellence (NICE) guidance. The pipeline runs on a single NVIDIA Jetson AGX Orin, ensuring that patient audio, transcripts, and outputs remain within institutional infrastructure. Evaluation included two recorded simulated MDT discussions, ten clinically validated synthetic discussions, and 1,270 acoustically augmented recordings. Optimisation of Whisper large-v3 reduced word error rate by 20.7% and 24.4% on the recorded discussions and achieved performance within 0.58% WER and 1.58% word information lost of a commercial clinical ASR benchmark on augmented audio. MedGemma-RAG identified 2.3 times more MDT-concordant interventions than a proprietary cloud comparator (p = 0.020), with no significant difference in overall accuracy. Stakeholders identified automated documentation, treatment recommendation support, and case triage as the most credible near-term applications while highlighting workflow integration, governance, and clinician trust as key implementation challenges. These findings demonstrate the feasibility of privacy-preserving, fully on-device AI for MDT documentation and guideline-informed decision support, providing a foundation for prospective clinical evaluation.
Background Radiation oncology workflows generate large volumes of electronic health record (EHR) data requiring daily synthesis. Large language model (LLM)-based automation is promising, but workflow-embedded implementations at scale remain limited. We describe the design and early usability and adoption evaluation of The Daily Dose (TDD), an LLM-driven system for automated clinical summarization and trial identification in radiation oncology. Materials and methods TDD delivers physician-specific email summaries each morning across three Mayo Clinic campuses using RadOnc-GPT (GPT-4o) to generate EHR-derived patient summaries and identify potentially eligible clinical trials for new or consult visits. One month post-deployment, an anonymous cross-sectional survey adapted from the System Usability Scale and Technology Acceptance Model was administered to all recipients. Results Fifty-five of 110 users responded (50%); 94.5% were in radiation oncology and 69.1% were attending physicians. Overall, 83.6% used TDD at least several times per week. Mean domain scores (5-point Likert) were 3.89 ± 1.04 for usability and satisfaction, 3.43 ± 1.24 for perceived usefulness, and 3.80 ± 1.17 for impact and future use. Satisfaction was significantly associated with perceived time savings (p < 0.001); 27% estimated saving ≥10 min daily. Internal consistency was high (α = 0.97). Free-text responses highlighted improved preparedness and patient-context awareness but noted occasional inaccuracies and imperfect trial matching. Conclusion In this early usability and adoption evaluation, a workflow-integrated LLM summarization tool was widely adopted and generally favorably perceived. These findings reflect user perceptions; objective validation of summary accuracy, trial-matching performance, and workflow efficiency is needed to establish clinical impact.
J. Holmes, F. Mastroleo, M. Borras-Osorio et al.· Clinical and Translational R...· 0 citations
Abstract Background Molecular Tumor Boards (MTBs) generate highly technical recommendations. The language used in their protocols is rarely accessible to patients. Lay-language patient protocols could support patient-clinician communication, yet manual production is difficult to sustain in high-volume oncology settings. Large language models (LLMs) may offer scalable drafting assistance, yet clinical usability remains largely uninvestigated under real-world deployment constraints. Existing evaluations rely predominantly on synthetic data or closed-source models that are incompatible with strict data protection requirements. Objective This study evaluated whether open-weight LLMs can provide clinically usable drafting support for German MTB patient protocols under real-world deployment constraints and developed a transferable evaluation framework for patient-facing text generation. Methods Eight open-weight LLMs were evaluated under zero-shot (A1) and one-shot (A2) prompting with constrained decoding, which ensures section-schema compliance. Automatic evaluation used ROUGE-1 (Recall-Oriented Understudy for Gisting Evaluation), BERTScore-F1 (Bidirectional Encoder Representations From Transformers Score), Wiener Sachtextformel version 4, and DistilBERT (Distilled Version of Bidirectional Encoder Representations From Transformers)–based complexity using a corpus of 316 MTB protocols and 47 expert-written patient protocols. For expert evaluation, 7 medical oncologists evaluated 50 protocols from the best-performing model across 3 International Organization for Standardization 9241‐11 usability dimensions using fine-grained error annotation, perceived postediting effort (PPEE), and net promoter score. Critical errors were defined as bearing the risk of patient harm. Results Llama-3.3-70B-Instruct achieved the strongest automatic performance. Across models, A2 significantly improved most automatic metrics compared to A1. However, expert usability evaluation of Llama-3.3-70B-Instruct showed the opposite picture: the proportion of protocols containing at least 1 critical error doubled under A2 (10/25, 40% vs 5/25, 20%) compared with A1, and the dominant error type shifted from language (40/108, 37%) errors to factual errors (69/145, 48%). Overall, 16% (230/1420) of the annotated paragraphs contained errors. Median PPEE was 2 (IQR 2.0‐3.0; low), and median net promoter score was 7 (IQR 5.0-9.0). Detractors (46/100, 46%) outweighed promoters (29/100, 29%), which suggests hesitation toward routine adoption. These differences in expert evaluation between A2 and A1 were directionally consistent but did not reach individual statistical significance for the paired samples (n=25). Conclusions Prompting strategies that improve automatic metrics can simultaneously increase the number of critical errors. Surface-level metric gains were, therefore, insufficient proxies for clinical safety. This was observed as a consistent directional pattern for a single model, but generalization to other models remains to be investigated. Nonetheless, the low paragraph-level error rate and favorable PPEE suggest that structured open-weight LLM generation may be a useful drafting support in a clinician-supervised setting. The proposed evaluation framework provides a text-quality-focused basis for future assessment of patient-facing LLM applications in real-world clinical settings.
T. Pakull, Noëlle Bender, Sven Benson et al.· Journal of Medical Internet...· 0 citations
Deploying artificial intelligence systems for medical image analysis in clinical settings involves considerations that go beyond model accuracy: infrastructure constraints, integration with existing workflows, and generalization across patient populations all determine whether a system works outside a research lab. This paper examines how computer vision architectures and large language models can be combined into a unified decision-support system for early detection of diseases through medical image analysis. Through a systematic review of 25 recent studies published between 2022–2025, the work develops an engineering-oriented taxonomy of architectures such as VGG16, DeiT, GPT-4 and LLaMA2, evaluating models against criteria including computational complexity, scalability, dataset dependency and deployment feasibility in low-resource environments. The evidence indicates that combining automated image analysis with LLM-based decision support can improve diagnostic accuracy and lower screening costs, with classification metrics exceeding 90% in controlled settings, contributing to the fulfillment of the Sustainable Development Goals. However, real-world deployment remains constrained by hardware requirements, interoperability gaps and dataset bias that limit generalization across diverse populations. This analysis provides concrete engineering guidelines for the design, validation and scalable implementation of AI systems in medical image analysis clinical workflows.
Humberto J. Navarro, M. S. González, Nubia Y. Piñeros et al.· Journal of Intelligent Decis...· 0 citations
Although promising, LLM-based systems are not yet reliable enough for autonomous medical diagnosis, and multiple recommendations for future research are contained to ensure a high level of safety, transparency, and clinical applicability for LLMs and other AI/ML-related technologies and devices.
M. U. K. Gunawardhna, Pirunthavi Wijikumar, D. Weerasinghe· Sri Lankan Journal of Applie...· 0 citations
Recent advances in large language models (LLMs) and vision transformers have enabled multimodal systems that integrate clinical text with medical imaging for diagnostic decision-making. While these systems show promising results on benchmark datasets in well-resourced research settings, their applicability in low-resource healthcare environments where diagnostic disparities are most severe remains limited and poorly understood. This mini review synthesizes key developments in LLM–vision fusion architectures from 2018 to 2026, with a focus on radiology-oriented visual question answering (VQA) and report generation systems viewed from a deployment perspective. Rather than comprehensively cataloguing multimodal medical AI, we synthesize the evolution of LLM–vision fusion architectures and discuss complementary deployment-enabling strategies, including parameter-efficient adaptation, post-training quantization, federated learning, and multilingual support, where they directly improve the feasibility of radiology AI in resource-constrained healthcare settings. Rather than focusing solely on performance benchmarks, we examine these approaches through a deployment-oriented lens, highlighting trade-offs between representational capacity, computational efficiency, interpretability, and memory footprint. We argue that current progress remains substantially shaped by model scaling and benchmark optimization, which often do not address the memory, connectivity, and annotation constraints of low-resource healthcare systems. While cross-modal transformer architectures provide strong representational alignment, their computational demands and reliance on large curated datasets limit real-world deployment. In contrast, emerging directions including parameter-efficient fine-tuning, post-training quantization, federated learning, and modular agent-based systems offer more tractable pathways toward clinical integration under hardware and data constraints. To bridge the gap between benchmark performance and clinical utility, we identify concrete challenges in data scarcity, multilingual coverage, and calibration, and propose a shift toward lightweight, interpretable, and hardware-aware multimodal AI. This perspective highlights the need to move beyond scaling-centric design toward models that can run on 4–8 GB VRAM, operate offline, and generalize across languages and imaging equipment.
Kahakashan Ashraf, Md.Hamid Hosen, N. Farah et al.· Frontiers in Digital Health· 0 citations
This narrative review examines the evolution of artificial intelligence (AI) in healthcare, with a focus on the transition from early rule-based systems to modern deep learning architectures and their integration into clinical practice. We examine foundational technologies, including convolutional neural networks for image interpretation, vision transformers for modeling long-range dependencies, and generative adversarial networks for image reconstruction and synthesis. The review further discusses the emergence of multimodal foundation models that integrate imaging with textual and genomic data to enhance diagnostic robustness. The application of these technologies is analyzed across three primary domains: Radiology (image enhancement and automated interpretation), cardiology (electrocardiographic and echocardiography analysis), and oncology (tumor classification and treatment planning). Specific attention is given to the national context in Türkiye, highlighting local initiatives such as TEKNOFEST and TÜBİTAK-supported projects that foster domestic AI development. While AI offers significant benefits in terms of diagnostic accuracy and treatment workflow optimization, challenges regarding data privacy, algorithmic bias, and interpretability (“black box” issues) persist. Future progress depends on the development of explainable AI, rigorous prospective validation, and the establishment of ethical regulatory frameworks.
Abdulkadir Yıldırım, Ö. Özdemi̇r· Artificial Intelligence in M...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
