Expert-rated accuracy and safety of patient-facing dental radiology responses from three AI platform configurations: a comparative evaluation

Abstract

Consumer-facing artificial intelligence platforms are increasingly used to obtain health information, but the quality of their responses to dental radiology questions requires evaluation. This study compared expert ratings of single responses generated by three selected consumer-facing AI platform configurations to common patient-facing dental radiology questions. Twenty expert-reviewed English-language questions were submitted using one standardized prompt to the investigator-recorded, user-visible ChatGPT-5.5 configuration with Thinking enabled, Gemini 3 Flash free configuration, and Microsoft Copilot 2026 free configuration during late April 2026. One response was collected from each platform for each question, producing 60 responses. Three Oral and Maxillofacial Radiologists independently rated coded responses with platform labels concealed, assessing accuracy, relevance, clarity, completeness, safety, triage appropriateness and misinformation. The 20 questions were treated as the primary paired experimental units. Evaluator-averaged scores were compared across platform configurations using Friedman tests followed by question-level exact paired comparisons, while individual evaluator ratings were used descriptively and for agreement assessment. The selected ChatGPT configuration received more favourable ratings than Gemini and Copilot for accuracy, relevance, safety and misinformation. It also received more favourable clarity and completeness ratings than Copilot. Gemini received more favourable question-level ratings than Copilot for clarity and completeness, but not for relevance. No significant differences were observed between ChatGPT and Gemini for clarity or completeness. Triage was applicable only to Question 20 and was reported descriptively. Considerable ceiling effects and tied ratings were observed, including 100% maximum ratings for ChatGPT accuracy and safety. Under the specified testing conditions, expert ratings differed across the single outputs generated by the selected platform configurations. The findings are limited by unequal platform configurations, one generation per question, unavailable historical interface metadata, a relatively homogeneous informational question set, ceiling effects, unverified masking, score–content inconsistencies and the absence of patient validation. They do not establish general model superiority, response repeatability, clinical readiness or patient benefit.