Trajectory Tracking Control for UUVs Based on Deep Reinforcement Learning

Abstract

Unmanned Underwater Vehicles (UUVs) operate in complex and uncertain environments, which require a suitable controller. While traditional PID controllers are widely used, they often have slow response speed and inadequate disturbance rejection, particularly under complex and uncertain conditions. To overcome these shortcomings, this paper introduces the DDPG-DLPID, an adaptive motion controller, including a Deep Deterministic Policy Gradient (DDPG) reinforcement learning that can acquire the parameters of PID controllers. In this paper, we design two loops: the inner loop handles velocity regulation, and the outer loop handles position and attitude. By using DDPG, the system can efficiently adjust the PID parameters of both loops in real time, allowing it to effectively adapt to environmental changes and achieve optimized requirements. To evaluate the controller, we design the following scenarios, including straight-line and complex path-following tasks. Compared with single-loop PID and dual-loop PID controllers, the proposed DDPGDLPID approach achieves faster response and higher tracking accuracy, while substantially reducing tracking errors under interference conditions. Physical experiments under three conditions-straight-line voyage, attitude maintaining, and depth control-were further carried out to validate the strategy’s real-world applicability. Experimental data confirm that DDPG-DLPID has better performance when compared with both traditional PID and dual-loop PID controllers across all test scenarios.