Hierarchical Federated DRL for Multi-UAV Path and Bandwidth Allocation in UAV-Assisted RAN

Abstract

With the development of AI-native radio access network (RAN), multi-unmanned aerial vehicle (UAV) cooperative wireless data collection based on multi-agent deep reinforcement learning (MADRL) has attracted wide attention. Existing studies rarely model the combined effects of multiple obstacles, interference, and dynamic changes in sensor node data. Furthermore, slow convergence and instability are observed in existing MADRL for UAV wireless communication. To address these issues, we proposes a hierarchical federated MADRL (HF-MADRL) framework that jointly optimizes UAVs’ trajectories and bandwidth allocation to achieve efficient data collection. Specifically, the problem is first formalized as a semi-partially observable markov decision process (SPOMDP) under the aforementioned complex scenario model, and a hierarchical decision-making architecture is designed to divide trajectory control and bandwidth allocation across different time scales. A hierarchical federated multi-agent deep deterministic policy gradient algorithm (HF-MADDPG) is then introduced to solve the problem. Finally, simulation results show that, compared with baseline algorithms, the proposed method attains significant improvements in data collection efficiency, training convergence speed, and stability.