PsyEval: a comprehensive large language model evaluation benchmark for mental health.
This work introduces PsyEval, a benchmark specifically designed to evaluate LLMs in mental health-related tasks across three core dimensions: knowledge, diagnosis, and emotional support, and reveals considerable gaps in LLMs' current ability to reason accurately and respond appropriately in mental health contexts.