Reference quality of OpenEvidence across five medical specialties

Abstract

OpenEvidence (OE) is a retrieval-augmented clinical AI tool. We evaluated the references OE returned to 150 standardized prompts across five specialties, analyzing all 4979 citations. No reference was confirmed as fabricated; three abstracts had attribution errors. Most references were recent and high-impact, though journal concentration varied by specialty. This study assessed reference existence and characteristics, not whether references support OE’s clinical claims - a benchmark for evaluating clinical AI tools.