Multi-View Contrastive and Semantically-Aligned EEG Emotion Recognition with Topology and Asymmetry-Aware Adaptation

Abstract

A reliable emotion recognition approach using EEG signals is crucial for affective human-computer interactions, but developing a reliable subject-independent decoder is very difficult. Despite previous research exploring either supervised deep learning or self-supervised contrastive approaches, no studies consider integrating neurophysiological structural knowledge into the problem via semantic supervision. This paper presents an integrated multi-objective framework incorporating focal classification, NT-Xent multi-view contrastive learning, and semantic alignment based on a language model. In addition to the two types of contrasts, we introduce the Topology and Asymmetry-Aware Multi-view Adapter (TAMA), which incorporates neurophysiological structural priors such as electrode topography, brain hemispheric asymmetry, and regional power spectral density into the backbone representations using topology-masked selfattention and gated residual fusion. Meanwhile, semantic supervision is conducted through the frozen BERT encoder during training, so our framework is deployable in an EEG-only mode. The experiments are performed on the DEAP dataset using an 8-fold leave-subjects-out cross-validation scheme. Our cnn_full_tama variant achieves a mean macro F1 of $0.475 \pm 0.033$, making it the best-performing model among all variants; interestingly, it is also the only model that shows positive mean MCC at +0.013 ± 0.082. Therefore, we explore the influence of different forms of inductive bias using the proposed framework. Contrastive learning appears to be critical for cross-subject generalisation, whereas TAMA boosts class-balanced discrimination.