While the present study is limited to short sessions with non-clinical participants in a controlled environment, NeuroVisionArm demonstrates the feasibility of a reproducible hybrid architecture for prosthetic command and target selection when EMG signals are compromised, serving as an engineering foundation for future longitudinal and clinical evaluation.
Abstract
Supporting fine motor interaction with upper-limb prostheses demands an interface intended to be intuitive, low-latency, and robust to user and signal variability. This challenge is particularly acute when EMG signals are unreliable, as in conditions such as ALS or high cervical SCI, where EEG-based BCIs offer a potential but challenging alternative due to low SNR and signal drift; gaze-only interfaces, in turn, suffer from the “Midas Touch” problem of unintended activations. Motivated by these target populations, we present NeuroVisionArm, a hybrid gaze–EEG system that couples fixation-gated target selection with lightweight MI–EEG command-state decoding mapped to Pick, Drop, and Idle states for semi-autonomous reaching and grasping in a controlled benchtop setting. In contrast to prior gaze–BCI systems, NeuroVisionArm introduces: 1) a fixation-gated late-fusion policy that explicitly time-aligns stable gaze fixations with MI windows using hysteresis and median filtering to suppress false triggers without long dwells; 2) a calibrated image-to-robot mapping that upgrades 2D gaze into metric 3D poses using MiDaS monocular depth and ArUco-based scale anchoring, while exposing pose uncertainty to the planner; and 3) an embedded EEG pipeline that combines CNN/LSTM/Random Forest models into an ensemble selected on a latency–accuracy Pareto frontier. We integrate these layers in a reproducible ROS/MoveIt stack and use complementary validation streams to separate physical task execution from decoder and fusion characterization. In 36 physical benchtop pick–place episodes, the system achieves a mean gaze localization error of $1.2 \pm 0.4$ cm, a planning success rate of 94%, and an overall grasp success rate of 87%; to keep EEG decoder validation independent of arm-task execution outcomes, the in-house MI–EEG model is reported as a separately controlled offline window-level metric of 92.1%. Additional validation includes external public MI-EEG benchmarking, public ERD/ERS physiology checks, RealSense RGB-D and ChArUco-anchored gaze-depth validation, and simulated fusion-policy sensitivity analysis using real gaze timing with EEG decisions parameterized from public-dataset operating characteristics. While the present study is limited to short sessions with non-clinical participants in a controlled environment, NeuroVisionArm demonstrates the feasibility of a reproducible hybrid architecture for prosthetic command and target selection when EMG signals are compromised, serving as an engineering foundation for future longitudinal and clinical evaluation.
A lightweight single-channel EEG-based BCI system that employs voluntary blink events as active control commands that provides a basis for trustworthy driver-intent decoding in intelligent cockpit applications is proposed.
Yu-Ming Nie, Fang Xu, Guo Wang et al.· IEEE Access· 0 citations
Dexterous hand control is essential for daily activities and a top restoration priority for people with tetraplegia, yet neurophysiological datasets suitable for studying and decoding naturalistic hand behavior remain scarce. Most available recordings constrain movements to a small number of degrees of freedom or cover...
Kai Liu, Shu-Ran Guan, Si-Zheng Luo et al.· bioRxiv· 0 citations
This Perspective argues that AI-mediated BCIs should be evaluated not only by decoding accuracy and task performance, but also by how AI assistance is deployed: when systems act autonomously, seek confirmation, preserve user effort or return control to the user.
This is among the first studies in which a wearable hybrid SSVEP-based BCI for lower-limb exoskeleton operation applicable to real-world walking tasks was systematically demonstrated and suggests that the proposed hybrid BCI is a robust and less physically demanding alternative to a conventional control method.
Jinuk Kwon, Hyojeong Han, Jihun Hwang et al.· IEEE transactions on neural...· 0 citations
Understanding language in real-world interaction requires methods that integrate auditory, visual, and neural signals. We present a proof-of-concept study combining mobile electroencephalography (EEG) and eye tracking to investigate referential selection during dyadic communication. Using a simplified version of the di...
Ingmar Brilmayer, Philip Georgis, P. Schumacher· Behavior Research Methods· 1 citation
A brain-computer interface (BCI) can detect neural activity related to movement, while a powered exoskeleton can provide the force needed to assist that movement. Electroencephalography (EEG) is noisy, muscle activity changes with fatigue and recovery, and the correct amount of assistance depends heavily on the person...
Hao-Heng Zhang· Theoretical and Natural Scie...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.