State-Based Reinforcement Learning Safe Trajectory Tracking for Autonomous Vehicles via Geometric Safety Representation and Lagrangian-PPO

Abstract

This paper investigates a state-based safe trajectory tracking (STT) control framework for autonomous vehicles in dynamic environments, where the objective is to design a controller that ensures accurate trajectory tracking while guaranteeing non-conservative collision avoidance. Existing learning-based approaches often suffer from inaccurate geometric safety representations and unstable training when incorporating safety constraints. To address these challenges, we propose a geometry-aware framework by integrating an analytical rotating supporting hyperplane (RSH)-based safety representation with a Lagrangian Proximal Policy Optimization (Lagrangian-PPO) reinforcement learning algorithm. By modeling vehicles and obstacles as rigid ellipsoids, an explicitly differentiable signed distance is derived to provide a continuous and non-conservative safety cost. Based on this representation, the trajectory tracking problem is formulated as a constrained Markov decision process, enabling unified optimization of tracking performance and safety via a reward–cost decoupled learning structure. Finally, asymptotic convergence and effectiveness of the proposed approach are validated through simulation and Vehicle-in-the-Loop (ViL) experiments, demonstrating superior tracking accuracy and robust obstacle avoidance compared with existing methods.