Real-to-real 2D crawling gait learning for a tendon-driven soft caterpillar robot

Abstract

Soft robots exhibit rich deformation and contact interactions that are particularly suited to crawling locomotion. At the same time, these properties make modeling and control challenging, especially in systems with continuous deformation and strong ground contact. While simulation-based reinforcement learning has been explored, transferring learned policies to physical systems often suffers from model inaccuracies. As a result, direct real-to-real reinforcement learning has attracted increasing attention, although existing demonstrations for soft crawling robots have largely been limited to one-dimensional sagittal-plane motion. In this study, we extend real-to-real reinforcement learning to two-dimensional planar crawling using an electrically driven tendon-wire soft robot. A soft caterpillar robot equipped with four independently actuated motors enabling twisting and lateral deformation was trained directly on the physical system for approximately 2.5 h. The learned policy enabled the robot to reach a target located about 500 mm away in approximately 50 s, while exhibiting diverse crawling behaviors. Successful goal reaching was also observed from randomly initialized starting positions. These results demonstrate that reinforcement learning can effectively exploit the complex deformation and contact dynamics of soft robots by directly operating a physical robot in the real world, enabling the generation of steerable crawling behaviors.