Safety-Critical Tracking Control of Self-Organizing Swarm for Unmanned Surface Vehicle Based on Identifier–Critic–Actor Reinforcement Learning
Abstract
In this article, a novel reinforcement learning-based distributed self-organizing swarm optimal control method is presented to address the challenges of real-time collision avoidance and optimal trajectory tracking for multiple unmanned surface vehicles (USVs) operating under uncertain environmental disturbances and in the absence of accurate dynamic models. To overcome the structural limitations of traditional formation methods during collision avoidance, a distributed self-organizing swarm mechanism with an all-round collision avoidance strategy is constructed, enabling the swarm to dynamically adjust its structure and behavioral modes in real time according to environmental changes and tracking tasks. To mitigate the unnecessary wide-ranging evasive maneuvers caused by a fixed obstacle avoidance radius, a velocity-regulation-based dynamic avoidance method is developed to reduce unnecessary path detours. Furthermore, to alleviate the control complexity arising from model uncertainties, high-frequency noise disturbances, and abrupt changes in system dynamics, an innovative identifier–critic–actor (ICA) synchronous learning neural network architecture based on a low-frequency (LF-ICA) learning identifier is proposed to achieve LF reconstruction while learning uncertain system dynamics, thereby ensuring the optimal consistency of the swarm posture during collision avoidance and trajectory tracking. Based on Lyapunov stability theory, it is shown that all signals in the swarm system remain bounded under the proposed control method. Finally, simulation results are presented to demonstrate the effectiveness of the proposed control approach.