Impact of Knowledge Source Type on RAG-Based LLMs in Specialized Medical Domains: A Case Study on Systemic Lupus Erythematosus
Factual reliability remains a critical bottleneck for Large Language Models (LLMs) in specialized medical domains like Systemic Lupus Erythematosus (SLE). While Retrieval-Augmented Generation (RAG) mitigates hallucinations, the qualitative impact of different underlying knowledge sources remains under-explored. This paper addresses this gap by evaluating three distinct repositories—Medical Textbooks, Academic Journals, and Clinical Guidelines—within a multi-path RAG framework using an open-weights, small-scale LLM (Gemma-3:4b). Based on a double-blind assessment of 45 stratified queries by three medical residents, we map the unique competency profiles of each source type. The core contributions of this study are twofold: (1) we provide an empirical characterization of medical source trade-offs, demonstrating that textbooks excel in foundational clarity and complex reasoning, guidelines ensure procedural precision, and journals provide technical depth at the cost of synthesis friction; and (2) we offer initial empirical insights that can help the development of future intent-aware RAG routing mechanisms to enhance clinical safety.