NeuroStream: spectral-spatio-temporal deep learning for visual stimulus classification from EEG

Abstract

Electroencephalography (EEG)-based visual classification is a challenging task due to low spatial resolution, complex temporal dynamics, and potential experimental confounds, yet with the recent advances in EEG classification, it offers a cost-effective, portable alternative with millisecond-level temporal resolution to Functional Magnetic Resonance Imaging (fMRI) for large scale studies and real-time applications. We propose a novel Spectral-Spatio-Temporal (SST) representation that transforms raw EEG signals into a structured, video-like format. Specifically, we compute wavelet transforms for all channels, aggregate log power into frequency bands, and map these features to electrode positions over time, thereby synthesizing the signal’s multi-dimensional dynamics into a unified, high-fidelity sequence. Building on this representation, we introduce the NeuroStream-SST framework, featuring a lightweight deep learning architecture optimized for spatiotemporal feature extraction. Experiments on the EEGCVPR40 dataset show that our approach reaches \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$71.25 \pm 0.25\%$$\end{document} accuracy in the high-gamma band using standard dataset splits, outperforming existing methods evaluated under an identical protocol and demonstrating its ability to capture complex neural characteristics effectively. Furthermore, we implement a set of evaluation protocols designed to expose and quantify the contribution of temporal correlations to reported accuracy. decoding performance declines steadily as the association between class labels and recording sessions is weakened, and falls to the majority-class baseline once the sessions of the evaluated classes are withheld entirely. These findings highlight our framework as a promising direction for EEG-based visual decoding, with implications for brain-computer interfaces and cognitive neuroscience.