Scooter-Driving Humanoid Robot: Sim-to-Real Transfer Through Deep Reinforcement Learning

Abstract

This paper presents a simulation-to-reality (sim-to-real) transfer process, allowing a full-sized humanoid robot to autonomously balance a two-wheeled scooter and track operator-supplied heading commands through a deep reinforcement learning (DRL) policy, including while carrying a human passenger for the first time. The learned policy governs the coupled balance and steering dynamics only: the heading reference and the target speed are supplied by a human operator, the throttle is set externally, and the initial launch over the first 2 to 3 m and the final stop are performed manually for safety. The system therefore performs autonomous balance and steering-command tracking during motion, rather than autonomous scooter driving. Unlike four-wheeled vehicles where the inherent stability simplifies control, scooter operation demands continuous and precise dynamic balancing coupled with real-time steering control, creating a challenging full-body control task. In this study, we demonstrate a successful integration of DRL techniques to bridge the sim-to-real gap, achieving stable closed-loop control of a humanoid robot balancing and steering a scooter under significant model uncertainty. A multi-scenario real-world evaluation shows the system maintaining control in the demanding low-speed regime of 0.8 to 3.0 m s−1, where balance control is most challenging and where speed varied across the range during trials rather than being held at a set point. Success rates are 0.96 on straight-line driving, 0.92 and 0.72 on light and sharp turns, 0.88 on the traversal of a 60 mm speed bump, and 0.73 while carrying a 70 kg passenger. They fall where sustained precise tracking is required, to 0.28 on a 4.5 m roundabout and 0.33 on part of the standardized Taiwanese scooter license test.