Use of a GenAI-patient for medical communication training for undergraduate medical students—a quasi-experimental study

Abstract

Effective communication is fundamental to high-quality healthcare. Communication skills training is a core component of medical curricula, yet opportunities to practice with standardized patients (SPs) are limited, costly, and can elicit anxiety in novice learners. Generative AI (GenAI) virtual patients may offer a scalable and accessible supplement. This quasi-experimental study examined the effect of practicing with a GenAI-patient on medical students’ self-reported nervousness, preparedness, self-efficacy, and self-perceived competence during their first SP encounter. First-year medical students ( n  = 383) at the University of Amsterdam were allocated on the basis of the class schedule to a control group receiving standard e-learning preparation ( n  = 182) or an intervention group offered voluntary additional access to a GenAI-patient built on a Retrieval Augmented Generation (RAG) framework using GPT-4o ( n  = 201). Students completed a 31-item questionnaire after their first SP class. Primary outcomes were nervousness, preparedness, self-efficacy, and self-perceived competence. Group differences were assessed using Mann–Whitney U tests with Benjamini-Hochberg false discovery rate (FDR) correction. The primary analysis was by intention-to-treat, comparing all students allocated to the intervention with all control students; the comparison restricted to actual users of the tool was secondary. Of 180 students that completed the questionnaire, 84 were controls and 96 students were allocated to the intervention, of whom 49 had used the GenAI-patient. No statistically significant differences were found between groups on the outcomes after FDR correction. In the intention-to-treat analysis; all composite effect sizes were near zero (Cliff's delta −0.05 to 0.04, all 95% confidence intervals including zero), as were those in the per-protocol comparison (−0.12 to 0.07). Among tool users, experience was generally positive: 63%–71% gave positive ratings. The majority considered the GenAI-patient a valuable contribution to the standard class preparation (71%). Voluntary, single-session text-based practice with a GenAI-patient did not significantly reduce nervousness or improve self-assessed communication competence before a first SP encounter. Positive user experience suggests potential for the format, provided future iterations incorporate mandatory minimum engagement, multiple patient cases, automated feedback, and higher-fidelity multimodal interaction.