TSMCA-Net: A Multi-Granular Cross-Attention Framework for Cross-Session EEG-fNIRS Decoding.

Abstract

We propose TSMCA-Net, a lightweight cross-session EEG-fNIRS decoder that achieves state-of-the-art accuracy with <1M parameters-70.6% fewer than the strongest baseline-while maintaining sub-8ms inference latency. TSMCA-Net decomposes multimodal interactions into three complementary views-temporal-spatial (TS), temporal-modal (TM), and spatial-modal (SM)-through structured multi-view factorization that separates heterogeneous dependency axes while preserving cross-branch information sharing. A Dynamic Multi-Scale Context module adaptively aggregates spatial dependencies on a unified scalp topology, while a causal lag modeling strategy captures delayed hemodynamic responses under strict temporal causality. An asymmetric cross-attention mechanism then integrates the multi-granular representations for role-aware fusion. Under a leave-one-session-out protocol on two public datasets, TSMCA-Net achieves the highest mean accuracy across three cognitive paradigms (71.41% MI, 86.36% MA, 80.30% WG), outperforming all 12 baselines in absolute accuracy and achieving statistical significance in 31 of 36 pairwise comparisons (p<0.05). Perturbation experiments demonstrate 38%-62% smaller accuracy degradation under simulated spatial perturbations.