Deep Reinforcement Learning-Based Control Strategies for Minimizing Interaction Forces in Lower-Limb Wearable Robots

Abstract

Wearable robots, particularly lower-limb exoskeletons, have gained attention for their use in rehabilitation and mobility assistance. A key challenge in their design is achieving natural, low-stress human-robot interaction with minimal discomfort. Zero-force control has emerged as an effective strategy for reducing contact forces and enhancing comfort. In this study a deep reinforcement learning (DRL)-based control method for a two-degree-of-freedom lower-limb wearable robot was developed to track natural walking motions while minimizing interaction forces and adapting to varying user characteristics. A simulation environment was created using MuJoCo, and the DRL agent was trained using the TD3 algorithm. Three control approaches were tested: direct velocity setpoint control, adaptive proportional-derivative controller tuning, and adaptive lead compensator tuning. Experimental results showed that the adaptive methods reduced average interaction forces from 3.1 N to 1.7 N (thigh) and from 7.2 N to 2.7 N (shank). The maximum forces during the walking experiments were also lower for the adaptive methods, with the third method achieving a maximum of 15 N. These results demonstrate that DRL-based methods, particularly when combined with traditional controllers, improve both force reduction and motion stability over conventional control strategies.