Hybrid Large Language Model-Reinforcement Learning Pipeline to Enhance Simulation Training for Robotics

Abstract

Despite rapid advances in artificial intelligence, robotic systems remain limited by poor generalisation across unstructured environments and fragile training pipelines. Reinforcement learning (RL) has shown promise in training robotics, yet its effectiveness is often constrained by manually engineered reward mechanisms. In parallel, large language models (LLMs) demonstrate strong reasoning and evaluation capabilities that remain underutilised in robotic training pipelines. This paper proposes a hybrid LLM-RL framework in which an LLM dynamically evaluates robot performance during simulation training and adaptively modifies the reward weights to improve learning stability, accuracy of task completion, and policy convergence. Unlike existing work that focuses on natural language control at inference time, the proposed method leverages the LLM during training, acting as a high-level reward critic. We implemented this framework using an open-source robotic arm trained in simulation to demonstrate improved task success rates and learning efficiency compared to static reward mechanisms. This work highlights a scalable pathway toward more adaptive and generalisable robotic training systems for advanced robotics.