Evaluating Large Language Model Localization for Clinical Psychology: Predicting Anxiety and Depression Scores in a Russian Student Sample

Abstract

Aim. To evaluate the suitability of large language models (LLMs), deployed in a closed-loop environment, for predicting anxiety and depression scores on the Hospital Anxiety and Depression Scale (HADS) in a sample of Russian students. Materials and methods. The empirical dataset consisted of de-identified responses from 946 first-year university students (669 female, 277 male) in Russia to 14 HADS items. For 11 models (DeepSeek-R1-32B, Qwen3-32B, T-pro-2.0/2.1, YandexGPT- 8B, GigaChat-20B, and others) deployed locally on GPUs, response probability distributions were extracted from token log probabilities. The experimental design included 5 prompt groups (textual instructions for the model), 2 presentation formats (structured dialogue and unstructured text), and 4 wording variations. Model-generated and empirical distributions were compared using Wasserstein distance (W); expected HADS subscale scores were additionally calculated. Results. T DeepSeek-R1-32B demonstrated the best accuracy in reproducing distributions (W = 0.293); however, it consistently underestimated expected anxiety scores (5.6 vs. 7.7). The most accurate expected anxiety scores were produced by YandexGPTpretrain (7.2) and T-pro-2.0 (7.1). Model selection explained 47% of metric variance; persona detail and respondent sex had no significant effect (p > 0.05). The depression subscale was reproduced less accurately than the anxiety subscale. Conclusions. Scores are robust to prompt variations: the level of detail in the persona description and the specific wording do not affect the outcome—a minimal description of the respondent suffices. Models trained in English favor English-language prompts, whereas for Russian-language models, language preferences are determined by the underlying architecture. The hypothesis regarding a systematic advantage for Russian-language models was not confirmed: quality is determined by domain adaptation rather than language. Practical application requires validation through experiments involving human subjects. The models do not replace clinical assessment by a specialist.