Multi-UAV-Aided Data Collection in Complex 3-D Urban Environments: A MADRL Approach Enhanced With Pheromone-Reward Shaping

Abstract

This paper investigates the problem of cooperative multiple unmanned aerial vehicles (UAVs) data collection for Internet of Things (IoT) networks in dense urban environments. Unlike existing studies that predominantly rely on idealized spatial models and average-based probabilistic channel models, this work explicitly accounts for realistic 3-D building distributions and deterministically models ground-to-air (G2A) channel blockages. We formulate a joint optimization problem to minimize the total task completion time, subject to stringent system throughput, flight dynamics, and energy constraints. To tackle the highly coupled challenges of node scheduling and trajectory planning, we propose a lightweight two-stage heuristic strategy for dynamic access control, along with a multi-agent reinforcement learning for trajectory planning. Crucially, to overcome the severe sparse-reward bottleneck inherent in complex 3-D obstacle avoidance, we introduce a Pheromone-based Reward Shaping (PRS) mechanism. By mathematically integrating the UAV’s kinematic state with deterministic environmental feedback, PRS effectively transforms the sparse-reward navigation challenge into a dense and smooth gradient, thereby profoundly accelerating policy convergence. Extensive simulations demonstrate that the proposed MATD3-PRS framework significantly outperforms representative baselines, achieving superior performance in task completion time, flight trajectory efficiency, and overall energy saving.