A Multi-UAV Cooperative Mission Planning Method Based on Multi-Agent Guided Soft Actor–Critic
TL;DR
A multi-agent guided soft actor–critic (MAGSAC) deep reinforcement learning algorithm to enable multiple UAVs to simultaneously arrive at multiple constant-velocity moving targets and outperforms existing mainstream algorithms in synchronization success rate, temporal synchronization accuracy, and safety.
Abstract
Multiple unmanned aerial vehicles (UAVs) performing cooperative missions in complex environments face challenges such as difficult cooperative decision-making, stringent spatiotemporal consistency constraints, and environmental uncertainty. The cooperative mission considered in this paper aims to enable multiple UAVs to simultaneously arrive at multiple constant-velocity moving targets. To address these challenges, this paper proposes a multi-agent guided soft actor–critic (MAGSAC) deep reinforcement learning algorithm. Under the centralized training with decentralized execution (CTDE) framework, a Guider network is introduced to guide the local actor network in learning coordinated strategies, thereby alleviating the non-stationarity of multi-agent decision-making under uncertain environments. An estimated time of arrival (ETA)-based spatiotemporal coordination reward function is designed to promote synchronized arrival. To address sparse rewards, a hindsight experience replay (HER) mechanism based on backward trajectory reconstruction is developed, and a delayed collision-constraint activation mechanism is incorporated to improve convergence while maintaining flight safety. Simulation results show that MAGSAC outperforms existing mainstream algorithms in synchronization success rate, temporal synchronization accuracy, and safety.