What and when: a Delphi study identifying AI competencies and learning objectives for medical students

Abstract

Introduction As Artificial Intelligence (AI) becomes embedded in clinical practice, medical graduates must be prepared to use it competently and critically. AI competencies and learning objectives (LOs) for medical students are needed. This study identifies these competencies and LOs for undergraduate medical education (UGME), particularly following the public release of large language models in 2022. Methods A four-round modified Delphi process was conducted between May 2024 and January 2025. The panel included medical students (n = 14) and medical professors (n = 2) in the first three rounds, and international experts in medical education and AI in the final round (n = 10). In Round 1, using an open-ended question, 77 potential LOs were identified. In Rounds 2 and 3, these items were rated using a 7-point Likert scale (1 = “Low Priority”, 7 = “High Priority”), and participants were provided an area for comments. Median scores and interquartile ranges were used to assess the strength of agreement, and the LOs were modified accordingly. In Round 4, validation input was sought from 10 international experts in medical education and AI. Learning objectives were organized into categories representing broader competencies. Results The process identified 7 competencies and 37 supporting LOs essential for medical graduates. Timelines for incorporating these across UGME were also identified as guidelines. Discussion This study offers a framework for integrating AI into UGME, combining theoretical understanding with ethical use to prepare future physicians for clinical practice. Developed with input from students and multinational AI expert stakeholders, these competencies and LOs constitute a student-partnered, internationally informed framework that may guide curricular development in other settings.