Domain Adaptation Techniques for Cross-Subject EEG-Based Emotion Recognition

Abstract

Electroencephalogram (EEG) signals used for emotion classification have gained a lot of research interest. However, improving the efficacy of emotion recognition across individuals is difficult. Due to the weak generalizability of characteristics across individuals, it has always been challenging to identify cross-subject emotions based on EEG. This study presents a novel deep learning-based method to recognize EEG-based emotions by capturing the concealed deep characteristics of EEG signals. This work proposes an approach to convert EEG signals into 2D spectrograms using Cross-wavelet transform (XWT) to feed a customized AlexNet model. The dense layers in the AlexNet are customized, where all the features obtained from the previous convolutional layers are combined. Further, the customized AlexNet model is fine-tuned with its parameters to learn emotions from input spectrograms. Then, the optimized AlexNet recognizes the emotions from the 2D XWT spectrograms and classifies them into respective emotion categories. We emphasized the attention maps of the convolution layers of the customized AlexNet model. Comprehensive simulations are conducted on three available datasets, the SJTU Emotion EEG Dataset (SEED), SEED-IV, and Dataset for Emotion Analysis of Physiological Signals (DEAP), to assess the performance of the presented technique. Moreover, the efficacy of the suggested approach is validated with cross-validation (CV) and cross-subject validation (CSV), whose outcomes are significant. Emotion classification performance is measured using accuracy, precision, recall, and F1 scores. The highest results achieved with the presented emotion classification framework for CV experiments are 98.76%, 97.37%, and 96.20% average classification accuracy (ACA), and for CSV experiments are 96.34%, 95.98% and 89.14% ACA with the respective SEED, SEED-IV, and DEAP datasets, i.e., higher than the published works.