Skip to content
Open access

NeuroVisionArm: Gaze-EEG Hybrid Control for Assistive Robotic Prosthetics

2026 · IEEE Access · Vol 14, pp. 133723-133742 · 0 citations · 51 references

TL;DR

While the present study is limited to short sessions with non-clinical participants in a controlled environment, NeuroVisionArm demonstrates the feasibility of a reproducible hybrid architecture for prosthetic command and target selection when EMG signals are compromised, serving as an engineering foundation for future longitudinal and clinical evaluation.

Abstract

Supporting fine motor interaction with upper-limb prostheses demands an interface intended to be intuitive, low-latency, and robust to user and signal variability. This challenge is particularly acute when EMG signals are unreliable, as in conditions such as ALS or high cervical SCI, where EEG-based BCIs offer a potential but challenging alternative due to low SNR and signal drift; gaze-only interfaces, in turn, suffer from the “Midas Touch” problem of unintended activations. Motivated by these target populations, we present NeuroVisionArm, a hybrid gaze–EEG system that couples fixation-gated target selection with lightweight MI–EEG command-state decoding mapped to Pick, Drop, and Idle states for semi-autonomous reaching and grasping in a controlled benchtop setting. In contrast to prior gaze–BCI systems, NeuroVisionArm introduces: 1) a fixation-gated late-fusion policy that explicitly time-aligns stable gaze fixations with MI windows using hysteresis and median filtering to suppress false triggers without long dwells; 2) a calibrated image-to-robot mapping that upgrades 2D gaze into metric 3D poses using MiDaS monocular depth and ArUco-based scale anchoring, while exposing pose uncertainty to the planner; and 3) an embedded EEG pipeline that combines CNN/LSTM/Random Forest models into an ensemble selected on a latency–accuracy Pareto frontier. We integrate these layers in a reproducible ROS/MoveIt stack and use complementary validation streams to separate physical task execution from decoder and fusion characterization. In 36 physical benchtop pick–place episodes, the system achieves a mean gaze localization error of $1.2 \pm 0.4$ cm, a planning success rate of 94%, and an overall grasp success rate of 87%; to keep EEG decoder validation independent of arm-task execution outcomes, the in-house MI–EEG model is reported as a separately controlled offline window-level metric of 92.1%. Additional validation includes external public MI-EEG benchmarking, public ERD/ERS physiology checks, RealSense RGB-D and ChArUco-anchored gaze-depth validation, and simulated fusion-policy sensitivity analysis using real gaze timing with EEG decisions parameterized from public-dataset operating characteristics. While the present study is limited to short sessions with non-clinical participants in a controlled environment, NeuroVisionArm demonstrates the feasibility of a reproducible hybrid architecture for prosthetic command and target selection when EMG signals are compromised, serving as an engineering foundation for future longitudinal and clinical evaluation.

Read PDF

Similar papers

Open access Oct 2026

Simultaneous multi-area recordings in macaque frontoparietal cortex during a manual object-interaction task

Dexterous hand control is essential for daily activities and a top restoration priority for people with tetraplegia, yet neurophysiological datasets suitable for studying and decoding naturalistic hand behavior remain scarce. Most available recordings constrain movements to a small number of degrees of freedom or cover...

Kai Liu, Shu-Ran Guan, Si-Zheng Luo et al. · 0 citations
#neuroscience Preprint Sep 2026

Slow-Fast Brain-Computer Interfaces: Preventing Neuroadaptive Overfitting in AI-Mediated Neural Interfaces

This Perspective argues that AI-mediated BCIs should be evaluated not only by decoding accuracy and task performance, but also by how AI assistance is deployed: when systems act autonomously, seek confirmation, preserve user effort or return control to the user.

Aarthy Nagarajan · 0 citations
Open access Sep 2026

Hybrid Brain–Computer Interface for Controlling a Wearable Lower Limb Exoskeleton With Augmented Reality Glasses for Gait Assistance

This is among the first studies in which a wearable hybrid SSVEP-based BCI for lower-limb exoskeleton operation applicable to real-world walking tasks was systematically demonstrated and suggests that the proposed hybrid BCI is a robust and less physically demanding alternative to a conventional control method.

Jinuk Kwon, Hyojeong Han, Jihun Hwang et al. · 0 citations
Open access Oct 2026

Combined eye tracking and electroencephalography during referential selection in dyadic interaction

Understanding language in real-world interaction requires methods that integrate auditory, visual, and neural signals. We present a proof-of-concept study combining mobile electroencephalography (EEG) and eye tracking to investigate referential selection during dyadic communication. Using a simplified version of the di...

Ingmar Brilmayer, Philip Georgis, P. Schumacher · 1 citation
Review Open access Sep 2026

Artificial Intelligence as a Decision Layer in Brain-Computer Interface-Controlled Exoskeletons for Rehabilitation and Human Augmentation

A brain-computer interface (BCI) can detect neural activity related to movement, while a powered exoskeleton can provide the force needed to assist that movement. Electroencephalography (EEG) is noisy, muscle activity changes with fatigue and recovery, and the correct amount of assistance depends heavily on the person...

Hao-Heng Zhang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.