Effective learning design guidelines for realizing arm-based locomotion on life-sized robotic hardware and expanding the traversable workspace of robots are provided.
Abstract
Brachiation is a form of locomotion in which primates move primarily using their arms, enabling traversal in environments without footholds. However, this motion requires highly coordinated whole-body movement and precise timing control for bar grasping and release. As a result, achieving robust behavior on life-sized robotic platforms remains challenging. In this study, we present a reinforcement learning-based method to realize brachiation on a life-sized dual-arm robot. The core of the proposed approach is Waypoint-Guided Reinforcement Learning (WGRL), a learning framework for inducing non-linear and complex motions. For high-difficulty tasks where imitation learning data are unavailable, WGRL guides behavior acquisition by sparsely specifying waypoints for the end-effector trajectory, while whole-body motion is generated through reinforcement learning. In addition, by integrating the waypoint-following guidance with rewards based on task success and mechanical energy, and training in an environment designed for Sim-to-Real transfer, the proposed method achieves both forward progression and motion stability. The acquired behavior is evaluated through Sim-to-Sim experiments under monkey-bar environments with geometric variations and hardware experiments, confirming robust brachiation including failure recovery behavior. This study provides effective learning design guidelines for realizing arm-based locomotion on life-sized robotic hardware and expanding the traversable workspace of robots.
This study proposes a new method, Wrench-Augmented Reinforcement Learning (WARL), which introduces a wrenche (force and torque) into the action space, and shows that introducing a wrench can encourage behaviors that do not sufficiently exploit the robot's physical embodiment.
Soft robots exhibit rich deformation and contact interactions that are particularly suited to crawling locomotion. At the same time, these properties make modeling and control challenging, especially in systems with continuous deformation and strong ground contact. While simulation-based reinforcement learning has been explored, transferring learned policies to physical systems often suffers from model inaccuracies. As a result, direct real-to-real reinforcement learning has attracted increasing attention, although existing demonstrations for soft crawling robots have largely been limited to one-dimensional sagittal-plane motion. In this study, we extend real-to-real reinforcement learning to two-dimensional planar crawling using an electrically driven tendon-wire soft robot. A soft caterpillar robot equipped with four independently actuated motors enabling twisting and lateral deformation was trained directly on the physical system for approximately 2.5 h. The learned policy enabled the robot to reach a target located about 500 mm away in approximately 50 s, while exhibiting diverse crawling behaviors. Successful goal reaching was also observed from randomly initialized starting positions. These results demonstrate that reinforcement learning can effectively exploit the complex deformation and contact dynamics of soft robots by directly operating a physical robot in the real world, enabling the generation of steerable crawling behaviors.
An integrated framework that combines a vision module for landing point and time prediction with a direct position and time conditioned RL locomotion policy, instead of intermediate velocity commands is proposed, which mitigates perception latency during dynamic interception.
Yi-Dong Zhu, Zibo Dai, Tong-Ning Zhang et al.· 0 citations
Quadruped robots can extend their utility beyond locomotion by using a leg as a non-prehensile end-effector to push objects, but this requires combining global planning with adaptive contact control. We present a hybrid framework that couples Rapidly exploring Random Trees (RRT) for global motion planning with Proximal Policy Optimization (PPO) for local leg-pushing control, evaluated in a CoppeliaSim simulation of a Spot-like quadruped pushing a box to a goal pose. The PPO action consists of Bézier control-point parameters and a leg-selection index, and the reward combines positional error, angular error, and a stability penalty. The agent learns straight-line pushing. Without retraining, a fixed asymmetric Bézier action induces a consistent rotation at ω≈4–5×10−4 rad/s, and the resulting circular arcs are composed—in the spirit of Dubins paths—to follow curved trajectories. For box masses from 0.1 kg to 0.9 kg (up to eight times the training mass), angular and position errors grew approximately linearly from 0.59° to 6.32° and 0.18 m to 0.64 m, respectively, with no abrupt divergence. A single learned pushing primitive, combined with sampling-based planning and a deterministic composition rule, generates both straight-line and curved manipulation.
This work proposes a hierarchical reinforcement learning pipeline that empowers the robots to perform aggressive locomotion through constrained obstacles--a narrow gate, extending the lifelike agility of legged robots to match that of their biological counterparts.
Zeren Luo, Jiahui Zhang, Yimin Han et al.· 1 citation
Context—Snake-like robots are biomimetic systems that can move effectively in narrow, complex, and restricted environments thanks to their modular and flexible body structures composed of numerous serially connected joints. These characteristics offer significant advantages, particularly in areas such as pipeline inspection, search and rescue operations, industrial maintenance applications, and exploration missions. The multiple degrees of freedom distributed along the body enable the robot to achieve high maneuverability but also make the control problem quite complex. Due to the dynamic interactions between segments, friction-based motion characteristics, and nonlinear system behavior, achieving reliable and accurate trajectory tracking emerges as a significant engineering problem.Objective—In this study, a reinforcement learning (RL) based control method has been developed to solve the trajectory tracking problem for snake-like robots in a two-dimensional plane.Method—In the proposed approach, the robot’s dynamic model was created in the Webots simulation environment, an open-source simulation program, and all training and testing processes were carried out in this environment. During the learning process, policy- based RL algorithms from the Stable-Baselines library were used. In this context, Proximal Policy Optimization (PPO) and three different RL algorithms were used during the training process. To enable the robot to adapt to different orientation scenarios, seven different angles defined in the range of +45 to −45 and trajectories of varying lengths were used. Thus, the goal was for the agent to learn a generalizable control policy not only for a specific trajectory type but also for tracks with different slopes and orientations.Results—The results obtained show that the PPO algorithm produced a higher average reward compared to other methods and exhibited a more stable learning process. After training was completed, the developed method was tested both on trajectories used during the training phase and on previously unseen trajectories. For the 0 trajectory, maximum errors were recorded as 0.093 m and 0.040 m for the x and y axes, respectively. Furthermore, the system exhibited robust generalization capabilities on a +22.5 trajectory, not encountered during the training phase, yielding maximum errors of 0.099 m and 0.052 m.Conclusion—These findings demonstrate that the proposed RL-based control approach can effectively solve the two-dimensional trajectory tracking problem in snake robots. In future studies, the proposed method can be extended to the three-dimensional trajectory tracking problem, or it can be evaluated under more complex conditions, such as scenarios involving obstacles.
Furkan Mezgil, M. Bingöl· Pamukkale Üniversitesi Mühen...· 0 citations
A weeklong summer workshop brought higher education faculty to campus to explore how AI and machine learning materials can be adapted for their classrooms.
