Velocity adaptive preview path tracking control for autonomous vehicles using deep reinforcement learning with Lyapunov function-based reward shaping

Abstract

Traditional rule-based path tracking controllers of autonomous vehicles (AVs) often struggle with nonlinear vehicle dynamics, parameter tuning, and changes in road conditions. Learning-based methods partly address these issues, but remain limited in adapting to velocity changes, exploiting the preview information from varying-curvature roads, and physically interpretable reward shaping that encourages Lyapunov function decrease. To address these challenges, a velocity adaptive preview soft actor–critic (APSAC) algorithm based on Lyapunov function-based reward shaping is proposed in this paper. Firstly, using a multi-point preview-based physical reference model for interpretability, longitudinal velocity is incorporated into the APSAC policy state and coordinates velocity adaptive preview-state construction, multiple reward parameters, and training with continuous acceleration and deceleration. Secondly, a basic reward mechanism is designed, which integrates state-dependent dense rewards with trigger-based rewards that drive state convergence. Building on this basic reward, a Lyapunov function-based potential-difference term is added to form the velocity adaptive hybrid reward. Under the stated assumptions, the conditional relationship between Lyapunov potential descent and basic reward maximization is examined. The convergence of idealized learning iterations is analyzed. Finally, simulation experiments and semi-physical hardware-in-the-loop (HiL) experiments demonstrate the superiority and engineering feasibility of the proposed method.