Objective- and Speed-Agnostic Human-in-the-Loop Reinforcement Learning Control of Hip Exoskeletons for Gait Assistance

Abstract

Wearable robotic lower-limb exoskeletons have demonstrated significant potential for providing personalized walking assistance tailored to individual needs. However, achieving efficient personalization through a unified control framework across different walking conditions and control objectives remains challenging for state-of-the-art human-in-the-loop optimization (HILO). In this paper, we developed a reinforcement-learning (RL)-based hierarchical control framework for robotic hip exoskeletons that unifies distinct control objectives into task-dependent state variables under multiple walking speeds and adjustable objective settings. The framework comprises three coordinated layers: a low-level integral-admittance-shaping controller to ensure reliable torque-tracking performance; a mid-level output-feedback-control-with-delay (OFCD) design that defines assistance shape without explicit gait phases; and a high-level model-free online least-squares policy iteration (LSPI) method initialized with a pre-trained policy for real-time OFCD parameter adaptation. Proof-of-concept treadmill walking experiments with ten healthy participants with artificially induced gait asymmetry and adjustable objectives, including spatial gait symmetry, peak hip flexion/extension (HFE), and step length, demonstrated individualized assistance on average within 5 minutes. Overall, spatial gait asymmetry was reduced to 0.2%, peak-HFE errors to less than 0.01 rad, and step-length error to 7.0 mm. These results support the feasibility of rapid, objective-dependent assistance personalization under controlled laboratory conditions and motivate future validation in clinical populations. Note to Practitioners—The practical significance of this work lies in enabling rapid, automated personalization of hip exoskeleton assistance without requiring manual tuning or explicit gait phase detection. Traditional HILO methods often involve time-consuming calibration and rely heavily on gait event segmentation, which limits adaptability in real-world use. The RL-based hierarchical control framework presented in this paper offers an efficient alternative by continuously adapting control parameters in response to user feedback and variations in walking speed. Practitioners developing assistive or rehabilitation devices may use this architecture as a reference for rapid parameter personalization under controlled treadmill walking conditions. The integral admittance shaping-based controller ensures stable torque delivery, while the mid-level coordination mechanism defines a low-dimensional assistance profile without explicit gait phase detection. The high-level online LSPI method provides real-time personalization across diverse control objectives, achieving individualized assistance on average within 5 minutes in proof-of-concept experiments on healthy participants with artificial gait asymmetry. Future studies should validate the framework in clinical populations before clinical or field deployment.