MULTI-UAV COORDINATED PATH PLANNING USING A MULTI-AGENT SOFT ACTOR-CRITIC ALGORITHM

Abstract

This work considers the issue of multi-agent coordination path planning for multiple UAVs with different properties working in a dynamic and uncertain environment. In order to resolve the stated problem, an algorithm based on multi-agent deep reinforcement learning called MultiAgent Soft Actor-Critic (MASAC) is suggested. An original simulation environment is developed to simulate a scenario of UAVs navigation towards certain goals, taking into account their kinematics constraints, obstacle avoidance and UAVs heterogeneity. As a consequence, the task is modeled as partially observable Markov decision process (POMDP). Performance assessment includes such metrics as task completion ratio, formation, flight efficiency and energy costs. The suggested framework for MASAC involves centralized training but decentralized execution which allows agents to coordinate actions and solve POMDP by means of local perception during testing time. As a result of experimental evaluation, an efficient way to resolve the curse of dimensionality, improve obstacle avoidance and cooperative formation control of UAVs was found. Such an approach shows great prospects of practical application in such domains as military operations, search and rescue missions, transport automation and disaster management.