Reinforcement learning-based fault-tolerant formation-surrounding control for UAVs-AUVs pursuit-evasion games under prescribed performance.

Abstract

This paper investigates the pursuit-evasion game problem for multiple unmanned aerial vehicles (UAVs) and autonomous underwater vehicles (AUVs) in the presence of actuator faults and performance constraints. The main contribution is the development of a reinforcement learning (RL)-based fault-tolerant formation-surrounding control (FSC) framework for multiagent systems under uncertain and constrained conditions. Firstly, a predefined-time observer (PTO) is designed to estimate lumped disturbances that include actuator faults and external disturbances. Then, a constrained position-loop subsystem for the multi-unmanned systems is constructed using a hyperbolic cotangent function, and the constrained system is further transformed into an unconstrained space via a hyperbolic tangent function. By utilizing the observer and the transformed position dynamics, an RL-based robust formation-surrounding control method is proposed. The efficacy and resilience of the proposed approach are illustrated using numerical simulations and 3D Simscape-based visualization simulations. Simulation results demonstrate that the proposed strategy achieves satisfactory transient performance and centimeter-level tracking accuracy under actuator faults and disturbances, with improved robustness and convergence performance.