MuJoCo-Based Sim-to-Real Reinforcement Learning for Transition Control of a Triple Inverted Pendulum

Abstract

This study presents an experimentally validated MuJoCo-based sim-to-real reinforcement learning framework for transition control among all eight equilibrium points (EPs) of a physical triple inverted pendulum (TIP). A multibody model is constructed from the apparatus properties, and uncertain actuation, damping, and friction parameters are identified by matching measured and simulated trajectories using the covariance matrix adaptation evolution strategy. The resulting effective parameter sets are sampled during training using truncated quantile critics to reduce policy dependence on a single model. The trained policies are transferred to the physical TIP without additional tuning or calibration. In the EP0-to-EP7 experiment, the policy trained without parameter identification failed in all ten trials, whereas the policy trained with the identified parameter sets succeeded in all ten. In the sequential experiment, all eight transitions were completed in each of ten trials, yielding 80 successful transitions with no observed failures while satisfying the cart travel and tangential force limits. These results demonstrate sim-to-real transition control across all eight EPs without state resets or precomputed trajectories.