A Multi-Source Retrieval-Augmented Large Language Model Architecture for Trustworthy Conversational AI in Diabetes Care

Abstract

Diabetes management requires continuous, personalized guidance for medication use, nutrition, glucose monitoring, and daily lifestyle decisions. However, many digital health tools cannot provide comprehensive support tailored to each patient’s medical condition and changing needs. Rule-based conversational systems can be safe but are often rigid and limited in natural interaction, whereas standalone large language models (LLMs) can produce fluent responses but may generate unsupported or clinically unsafe information, lack clear source attribution, and cannot reliably use patient-specific information or updated medical knowledge. Other challenges include limited personalization and fragmented diabetes knowledge distributed across clinical guidelines, patient records, nutritional databases, glucose-monitoring data, and medication safety resources. To address these limitations, this paper presents a Retrieval-Augmented Large Language Model (RA-LLM) architecture for trustworthy conversational AI in diabetes care which is a part of our AI-based Diabetes Care (AIDCare) mHealth solution. The proposed framework integrates clinical guidelines, structured electronic health record attributes, nutrition knowledge, optional glucose-monitoring context, and pharmaceutical safety rules. It combines clinical entity recognition, hybrid dense–sparse retrieval, patient-context filtering, cross-encoder reranking, evidence-constrained prompt construction, and multi-layer safety verification to generate personalized, evidence-supported conversational responses. The proposed system was evaluated on a controlled benchmark of 10,000 synthetic diabetes interaction scenarios covering medication guidance, nutrition, glucose monitoring, complication awareness, and lifestyle management. Compared with a vanilla LLM baseline, the RA-LLM improved evidence-grounded response correctness from 78.3% to 94.7%, increased source attribution reliability from 34.0% to 94.0%, and reduced the predefined safety-violation rate by approximately 89%. These findings show the potential of multi-source retrieval, patient-context integration, and safety-aware conversational generation for improving AI-assisted diabetes self-management. As the evaluation is based on synthetic scenarios, real-world clinical validation remains necessary.