GA-assisted Multi-Agent Reinforcement Learning for UAV Relay Network Deployment Optimization

Abstract

In post-disaster communication scenarios, the three-dimensional (3D) optimal deployment of unmanned aerial vehicles (UAVs) acting as relay nodes becomes intractable in large scale environments. Multi-agent reinforcement learning (MARL) often fails to converge, resulting in challenges such as erratic flight paths and unstable network topologies. This paper proposes a genetic algorithm (GA)-assisted multi-agent proximal policy optimization (MAPPO) (GA-MAPPO) framework that uses a GA to provide initial UAV deployment positions for MAPPO training and introduces the Hungarian algorithm (HA) to optimally assign the positions generated by the GA to UAVs regardless of numerical mismatches. Simulations using 13 UAVs in three main orientation configurations demonstrated that the proposed GA-MAPPO achieved faster convergence and higher rewards than conventional MAPPO. Moreover, with the number of episodes fixed and the number of UAVs varied from 1 to 15, GA-MAPPO reduced the converged reward penalty by approximately 75% (from about −0.32 to −0.08) and shortened the computation time by roughly 10% compared with conventional MAPPO, validating the proposed hybrid framework for large-scale 3D millimeterwave (mmWave) relay deployment.