Across medical benchmarks, MedLLM shows a pattern visible only at sub-billion scale: medical competence does not degrade uniformly under compression but splits by task type and dissociation is masked at 7B, where both capabilities are present, and surfaces only when capacity is scarce.
Abstract
Open medical language models have converged on a single scale: every widely used system runs at 7B parameters or more, leaving the sub-billion regime uncharacterized. We present MedLLM, an open 0.1B-parameter medical language model trained through a fully open three-phase pipeline: general pretraining with curriculum sequence-length scheduling, domain fine-tuning on MedFineWeb, a reference-guided medical corpus we release that is selected from general web data by embedding similarity to medical question-answering (QA) data, and preference-aligned fine-tuning combining SFT with direct preference optimization (DPO). Across medical benchmarks, MedLLM shows a pattern visible only at sub-billion scale: medical competence does not degrade uniformly under compression but splits by task type. On context-grounded QA it comes within $2.9$pp of a medically adapted 7B model and surpasses the instruction-tuned and general-purpose 7B baselines; on knowledge-recall QA it stays near the task floor on clinical-vignette MedQA yet significantly exceeds every 7B and sub-7B baseline on MedMCQA, indicating that where recall fails the constraint is model capacity rather than adaptation. This dissociation is masked at 7B, where both capabilities are present, and surfaces only when capacity is scarce.
An engineering-oriented, end-to-end roadmap that structures the full lifecycle of clinical language model systems—from model design and domain adaptation to optimization and real-world evaluation is introduced.
This entry-level tutorial aims to equip healthcare professionals with the tools necessary to effectively integrate LLMs into clinical practice, ensuring that these powerful technologies are applied in a safe, reliable, and impactful manner.
Qiao Jin, Nicholas Wan, Robert Leaman et al.· Nature Protocols· 1 citation
In this study, we introduce a novel benchmark comprising over 9,000 real-world, point-of-care, multilingual, and multimodal clinical question-answer pairs sourced from frontline health workers in Nigeria. Using the dataset, we compare local general practitioners to multiple leading open and closed LLMs. Our results reveal several critical insights into the suitability of LLMs as clinical decision support systems in low-resource contexts. The results confirm that performance varies widely by language and input modality (e.g., text vs speech): while models perform best on English text inputs, their accuracy drops significantly for local-language speech. Critically, it is possible to achieve substantial performance gains by transcribing and translating other languages into English before prompting an LLM-- an important insight for non-anglophone product developers. Finally, this benchmark highlights key limitations of SLMs in supporting frontline healthcare in low-resource settings and provides a clear opportunity to track improvements as novel solutions are developed.
Tobi Olatunji, C. Aka, C. Okocha et al.· medRxiv· 0 citations
Developing artificial intelligence capable of clinical language comprehension and reliable diagnostic reasoning has remained a core challenge in biomedical engineering. While Large Language Models (LLMs) demonstrate significant potential in general natural language processing tasks, their direct application in the medical domain is severely constrained by parametric hallucinations and data silos. This paper introduces an end-to-end, resource-efficient, multilingual speech-driven Question-Answering (QA) framework optimized for localized clinical support. To accommodate deployment on consumer-grade execution environments, we implement Parameter-Efficient Fine-Tuning (PEFT) using Low-Rank Adaptation (LoRA) and 4-bit Quantized LoRA (QLoRA) configurations across open-source 3B and 7B parameter architectures. Human preference alignment is enforced via a stateful Reinforcement Learning with Human Feedback (RLHF) loop applying Proximal Policy Optimization (PPO). Crucially, to mitigate the vulnerabilities of passive information retrieval, we introduce an Active Validation Loop powered by Corrective Retrieval-Augmented Generation (CRAG). This validation engine is decoupled from the model harness using the Model Context Protocol (MCP), standardizing asynchronous lookups across dense vector repositories, clinical guidelines, and real-time electronic health registries.
Misha Patel· International Journal of Sci...· 0 citations
The integration of artificial intelligence into medical question-answering systems has advanced rapidly; however, research remains predominantly focused on English, leaving low resource languages like Persian significantly underserved. To address this gap, this paper introduces Gaokerena, a novel family of compact Persian medical language models optimized for deployment on consumer grade hardware. As a foundational step toward localized digital healthcare, we first present Gaokerena-V, developed by training a baseline model on a newly curated 90-million-token Persian medical corpus and 20,000 expert-vetted physician Q&A pairs, which improved performance on a translated medical MMLU benchmark from 46.28% to 49.31%. Second, recognizing the critical demands of clinical reasoning, we developed Gaokerena-R by integrating a Chain-of-Thought approach with two novel Reinforcement Learning with AI Feedback (RLAIF) frameworks to optimize preference-based reasoning. Despite utilizing the same baseline architecture and a smaller dataset than Gaokerena-V, Gaokerena-R achieved a superior benchmark score of 52.98%. Furthermore, both models are equipped with custom-developed uncertainty heads that predict the model's confidence in its responses based solely on internal hidden states. While these results demonstrate significant progress in Persian medical language modeling and proactive safety estimation, current performance levels remain insufficient for direct clinical application, highlighting the necessity for further research into robust knowledge acquisition and rigorous safety verification prior to real world deployment.
It is suggested that domain-adapted encoder models may be preferable for similar structured clinical NER settings, although larger and externally validated benchmarks are needed before generalizing to other languages, clinical corpora, model families, or deployment environments.
L. Elvas, Carolina Carvalho· Scientific Reports· 0 citations