Incremental Context Expansion and Domain-Adapted Transformers for Early Mental Health Risk Detection in Spanish

Abstract

Early detection of mental health disorders in Spanish is limited by two main factors: the scarcity of domain-specific Spanish language models and the difficulty of making timely predictions from long, progressively available social media histories. This paper addresses these challenges through four main contributions. First, we introduce three Spanish foundational models specifically adapted to the mental health domain through domain-adaptive pre-training. Second, we propose Incremental Context Expansion (ICE), an automatic dataset relabeling methodology designed for early detection. ICE identifies the point at which cumulative messages provide enough evidence of a disorder, generating more informative training samples. Third, we formally characterize ICE and evaluate its effect using bootstrap confidence intervals and paired significance tests. Fourth, we provide a set of fine-tuned models trained on samples generated using the ICE methodology for early risk detection tasks. Experiments on three Spanish MentalRisk benchmarks (MR23-ED, MR23-D and MR24-DD) show that ICE improves early-detection behavior, especially when models must make decisions from limited initial evidence. Compared with published MentalRisk shared-task systems, ICE models achieve competitive results across the evaluated benchmarks and obtain the best reported values in several task–metric combinations, particularly in MR24-DD and MR23-D. In MR23-ED, where classical lexical models remain highly competitive, ICE reduces the gap with the strongest reference systems while improving strict early-detection behavior. These findings indicate that ICE reduces the dependency on fixed minimum-context thresholds while maintaining strong classification performance. All models are publicly available.