Beyond Surveys: Predicting Student Well-being from Open-ended Video Responses using LLMs

Abstract

Traditional well-being assessment relies on infrequent self-report questionnaires (e.g., PHQ-9), which provide only sparse, retrospective snapshots of well-being and may miss everyday fluctuations. This creates an opportunity for low-burden, lightweight assessments embedded in everyday interactions, such as open-ended conversations with AI systems, to complement existing approaches and enable more natural, potentially more reliable monitoring of well-being over time. However, it remains unclear whether large language models (LLMs) can predict self-reported psychological well-being from naturalistic multimodal data. As a first step toward such an AI-based informal assessment system, we recruited N=228 college students to collect self-reported well-being responses paired with corresponding open-ended video responses. Using this dataset, we conducted four studies to examine the effects of input modalities, model architectures, contextual inputs, and prompting strategies. Our results show that LLMs leveraging multimodal features outperform regression baselines without task-specific fine-tuning, and that retrieval-augmented chain-of-thought prompting with contextual inputs yields the most robust performance. These findings show the promise of multimodal LLM-based approaches for scalable, low-burden well-being assessment and offer design guidance for integrating conversational AI into real-world affective computing systems.