Skip to content
Review Open access

Development, validation and AI chatbot evaluation of the Adolescent Mental Health AI Response Tool

Oct 2026 · Nature Mental Health · 0 citations · 22 references
Medicine

Abstract

Many adolescents seek mental health advice from general-purpose artificial intelligence (AI) chatbots, despite limited evaluations against youth-relevant clinical standards. We developed the Adolescent Mental Health AI Response Tool (AMHART), a clinician-informed instrument for auditing the presence of guideline-informed response content across seven domains, and applied it to four major AI chatbots. Candidate items were reviewed by clinicians for their evidence base, objectivity, and clinical importance, yielding a subset with strong content validity and inter-rater reliability. We then configured a three-model large language model evaluator ensemble, assessed its agreement with human reference labels, and scored multi-turn conversations in which adolescent personas (varying by sex, diagnosis, and symptom severity) engaged with ChatGPT-5, ChatGPT-4, Claude-4.5, and Gemini-2.5. AMHART scores indicated greater coverage of applicable criteria for anxiety than depression and for moderate than severe presentations. Findings suggest AMHART offers a reliable, clinician-informed benchmark for evaluating chatbot responses to adolescent anxiety and depression disclosures.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.