Multi-UAV Trajectory Planning for Dynamic Target Search: An LLM-Enhanced Multi-Agent Reinforcement Learning Algorithm

Abstract

Deploying Uncrewed Aerial Vehicles (UAVs) for dynamic target search in disaster response scenarios can reduce losses. This paper investigates multi-UAV cooperative trajectory planning for dynamic target search in a three-dimensional environment with static obstacles, aiming to maximize the number of searched targets and minimize the average uncertainty of the search area, while ensuring collision avoidance between UAVs and obstacles. Existing Multi-Agent Reinforcement Learning (MARL) based methods face the sparse reward problem in dynamic target search, which hinders planning feasible multi-UAV trajectories. Notably, Large Language Models (LLMs), with extensive pre-trained knowledge and powerful semantic reasoning capabilities, exhibit potential for designing high-quality reward functions to alleviate the sparse reward problem. Therefore, we propose an LLM-guided Multi-Agent Proximal Policy Optimization (LLM-MAPPO) algorithm, which leverages LLMs’ reasoning capabilities to guide MARL policy learning and plans multi-UAV trajectories for efficient dynamic target search. Specifically, we design an offline LLM reward shaping scheme that generates dense reward signals to mitigate the sparse reward problem. Moreover, we propose a dual-mode pheromone-based search mechanism to guide UAVs to respond promptly to changes in target positions. Experimental results demonstrate that LLM-MAPPO significantly outperforms compared algorithms in terms of the number of searched targets and average area uncertainty, while successfully avoiding collisions. In particular, LLM-MAPPO reduces the target search time by 71.4%.