Generative small language models in clinical NLP: applications, adaptation, and evaluation

Abstract

Large Language Models (LLMs), such as Generative Pre-trained Transformer 4 (GPT-4) and Medical Pathways Language Model (Med-PaLM), have transformed clinical Natural Language Processing (NLP), demonstrating remarkable capabilities in understanding and generating medical text. However, their deployment in healthcare remains constrained by high computational demands, data privacy risks, and limited interpretability. In response, this survey provides a comprehensive synthesis of research on Small Language Models (SLMs) in medicine, introducing an engineering-oriented, end-to-end roadmap that structures the full lifecycle of clinical language model systems—from model design and domain adaptation to optimization and real-world evaluation. We systematically reviewed lightweight architectures, parameter-efficient adaptation strategies, and data-centric training techniques that enable domain-specific optimization under limited computational resources. The survey organizes SLM applications across core NLP tasks—classification, relation extraction, summarization, and question answering—and further examines specialized models designed for domains such as radiology, oncology, ophthalmology, nuclear medicine, and cardiology. A multidimensional evaluation framework was introduced, combining quantitative and qualitative metrics to assess factual accuracy, clinical reliability, and computational efficiency. Finally, we identify ongoing challenges and future directions, emphasizing Retrieval-Augmented Generation (RAG), Reinforcement Learning from Human Feedback (RLHF), and agentic Artificial Intelligence (AI) frameworks that enable language models to support planning, tool use, and workflow-aware clinical decision-making. Collectively, this study establishes a foundation for engineering-driven SLM pipelines that balance efficiency, adaptability, and clinical trustworthiness in real-world healthcare applications. This review is intended for a broad interdisciplinary audience, including clinical NLP researchers, machine learning practitioners, healthcare AI engineers, biomedical informatics specialists, and clinicians seeking practical guidance on deploying efficient and privacy-preserving language technologies.