Prediction in routine care: a naturalistic comparison of clinicians and a machine learning model for patients receiving treatment for generalised anxiety disorder.

Objective

Approximately half of patients with generalised anxiety disorder (GAD) do not recover following psychological treatment. Accurate prognostic judgement may support clinical decision-making and planning. This study compared the predictive accuracy of clinicians and a validated machine learning model and examined factors clinicians reported using when forming predictions.

Method

Clinicians from two routine psychological therapies services in the UK predicted post-treatment GAD-7 scores and categorical outcomes (reliable improvement, reliable recovery, deterioration) for 100 patients prior to treatment, providing confidence ratings and free-text explanations. Predictions were compared with those generated by a pre-trained Bayesian Additive Regression Tree model using mean absolute error (MAE), root mean squared error (RMSE), and area under the receiver operating characteristic curve (AUC). Clinician rationales were analysed thematically.

Results

Among 66 patients with complete outcomes, the model showed numerically greater accuracy for post-treatment GAD-7 scores, reliable improvement, and recovery, although these differences were not statistically significant. Both approaches performed poorly for deterioration. Clinicians reported high confidence (M = 67.3%), which was unrelated to accuracy. Thematic analysis identified contextual factors such as motivation and engagement not captured in routinely collected data.

Conclusion

Structured prediction models may enhance prognostic accuracy in routine psychotherapy and complement clinician judgement.