A multimodal EEG-based psychological risk representation method using spatiotemporal graph attention and adaptive gated fusion
TL;DR
A cross-modal temporal representation method that integrates spatiotemporal graph attention with an adaptive gating mechanism that significantly improves the accuracy and robustness of risk prediction on the AMIGOS dataset, effectively overcoming the subjective interference of questionnaire-based assessments.
Abstract
As mental health issues gain increasing attention, traditional risk assessment methods based on psychological questionnaires are limited by subjective biases and a narrow dimensionality of information, making them difficult to meet the needs of precision medicine. To address this challenge, this study proposes a cross-modal temporal representation method that integrates spatiotemporal graph attention with an adaptive gating mechanism. The method aims to dynamically correct and provide explainable analysis of the health risk labels generated from psychological profiling using objective electroencephalogram (EEG) signals. Specifically, the study first constructs individual psychological profiles based on the raw Big Five personality scores from the AMIGOS public dataset and generates initial health risk labels in combination with self-reported emotional data. Subsequently, for the 14-channel EEG signals, a cascaded architecture combining graph attention networks and bidirectional long short-term memory networks is designed to effectively extract spatial topological features and temporal dynamic information from brain regions. Finally, a cross-modal attention mechanism and a temporal gating fusion module are introduced to deeply integrate static psychological features with dynamic neural signals, thereby refining the representation of risk labels. Experimental results demonstrate that the proposed method significantly improves the accuracy and robustness of risk prediction on the AMIGOS dataset, effectively overcoming the subjective interference of questionnaire-based assessments. Additionally, through quantitative analysis of attention weights, this study reveals the contribution patterns of key EEG features to risk prediction, providing an objective basis for personalized psychological intervention strategies and offering a new technical pathway for low-cost, interpretable multimodal mental health assessment.