SmartMM: A Domain-Specific Large Language Model for Medical Microbiology

Abstract

Background: Large language models (LLMs) show considerable promise for medical question answering and reasoning. Their use in medical microbiology, however, remains constrained by limited domain-specific knowledge and the risk of hallucinated outputs. Objective: To develop and evaluate Smart Medical Microbiology (SmartMM), a specialized LLM for accurate, reliable, and context-aware responses in medical microbiology. Methods: SmartMM integrates domain-adaptive continual pretraining, supervised fine-tuning (SFT), reinforcement learning from human feedback (RLHF), knowledge distillation, and retrieval-augmented generation (RAG). We constructed a high-quality microbiology corpus from textbooks, clinical guidelines, the scientific literature, case reports, and other authoritative sources. Model performance was assessed using objective examinations, subjective generation tasks, expert review, and real-world user preference evaluation. Results: SmartMM achieved accuracies of 0.897 and 0.563 on true-or-false and fill-in-the-blank questions, respectively. In subjective generation tasks, it obtained the highest ROUGE-L score (0.265) and BERTScore F1 score (0.771) among all compared models. Expert assessment showed excellent inter-rater reliability, with all ICC(C,3) values exceeding 0.970. In a user evaluation involving 20 participants and 100 real-world questions, SmartMM received the largest number of first-place rankings (33), placing it among the top-performing systems overall. Conclusions: SmartMM showed strong domain adaptability in medical microbiology knowledge organization, semantic generation, and retrieval-augmented reasoning. These findings support its potential use in educational support, infectious disease knowledge assistance, and retrieval-enhanced medical question answering.