Enhancing Human-SRL Collaboration: A Vision-Based Integrated Control Framework for Trajectory Prediction and Automatic Load Compensation
Supernumerary Robotic Limbs (SRLs) are designed to augment human manipulation capabilities. However, achieving efficient and safe human-robot collaboration (HRC) in real-time scenarios with varying loads remains challenging. This paper proposes a vision-based collaborative control framework integrating human intention recognition and adaptive gravity compensation. First, a vision-based motion tracking algorithm captures hand positions and gestures in real-time to interpret human intentions. Second, a trajectory prediction method based on an autoregressive model is proposed, which incorporates virtual interaction force prediction to improve the response speed and accuracy of the SRL in motion tracking. Finally, an enhanced gravity compensation algorithm is introduced that utilizes real-time inertial measurement unit (IMU) quaternion data for base posture correction and estimates end-effector load online through force-torque relationships, enabling automatic adaptation to unknown and continuously varying loads without manual parameter configuration. Experimental results demonstrate that the proposed framework reduces trajectory tracking errors by 21.49%, 55.40%, and 37.79% in the X, Y, and Z directions, respectively, compared to baseline methods through enhanced human intention recognition. In a dual-hand coordination task where the operator guides the SRL with one hand while performing auxiliary operations with the other, the framework achieves 62.42% and 33.27% reductions in Z-axis displacement and total movement distance under a continuous load increase of approximately 133%. Multi-participant validation across five participants further confirms the generalizability and consistency of the proposed framework. Note to Practitioners—This paper presents a vision-based framework for enhancing human-robot collaboration with SRL in real-time scenarios involving varying loads. The framework addresses identified practical challenges including trajectory tracking accuracy and system stability under dynamic conditions. It is applicable to manufacturing assembly, collaborative transportation, and precision operations. Through intuitive motion tracking via hand gesture recognition, the approach enables natural human-robot interaction without requiring complex user training. The combined position and force prediction mechanism improves trajectory following performance, while the IMU-enhanced gravity compensation enables configuration-free adaptation to base posture changes and varying loads, which is valuable for practical wearable SRL applications.