Explainable Reliability-Gated EEG–EMG Fusion for Offline Early Gesture and Intention Recognition with Potential Relevance to Human–Robot Collaboration

Abstract

Early gesture and intention recognition from biosignals may provide a useful perception capability for future human–robot collaboration systems. This study proposes and evaluates an explainable reliability-gated EEG–EMG fusion framework for offline classification of seven upper-limb hand gestures and associated instructed intentions. The framework uses modality-specific EEG and EMG encoders followed by an adaptive reliability gate that learns how much each biosignal modality should contribute to the final prediction. The model was evaluated on synchronized EEG–EMG upper-limb gesture data using leakage-free trial-level splitting, multi-seed evaluation, classical and deep learning baselines, ablation variants, early-prefix testing, uncertainty-aware selective prediction, Integrated Gradients explainability, leave-one-subject-out validation, and few-shot adaptation. The results show that EEG–EMG fusion improved trial-level Macro F1 from 0.3566 ± 0.0217 for EEG-only and 0.7117 ± 0.0870 for EMG-only to 0.7985 ± 0.0178. Under the primary development/validation trial-level split, the final reliability-gated model achieved 0.8204 ± 0.0234 trial-level Macro F1 and 0.7238 ± 0.0151 window-level Macro F1. Because this validation subset was also used for training control and checkpoint selection, these values represent internal validation estimates rather than independent test-set performance. Under leave-one-subject-out evaluation, trial-level Macro F1 decreased to 0.4411 ± 0.3654, demonstrating weak and highly variable calibration-free generalization to unseen users. Early-prefix evaluation showed useful prediction after 2 s and near-complete performance after 5 s. Uncertainty-aware selective prediction increased accuracy to 0.9012 ± 0.0107 at 70% coverage. Integrated Gradients attributed 69.15% of the model-level decision evidence to EMG and 30.85% to EEG, indicating that the trained model relied more strongly on EMG features under the evaluated dataset while retaining a complementary contribution from EEG. These findings demonstrate the potential of the proposed reliability-gated framework for explainable offline EEG–EMG intention recognition on the evaluated benchmark dataset while highlighting the need for validation on larger and more diverse subject populations before broader generalization to practical HRC applications.