Deep Reinforcement Learning-Based Service Offloading Optimization in 6G FANETs

Abstract

6G flying ad-hoc networks (FANETs) are envisioned to effectively enlarge the traditional edge computing coverage. However, the requirements of high energy efficiency and low latency seriously constrain the FANETs performance. In this paper, we investigate a deep reinforcement learning (DRL) scheme for optimizing the service offloading cost in FANETs, where the mobile services can be intelligently offloaded in two layers between terrestrial users (TUs), unmanned aerial vehicles (UAVs) and macro base stations (MBSs), respectively. Specifically, a multi-agent (MA) deep deterministic policy gradient (DDPG) is employed to jointly control the power and lower the offloading latency. Due to the randomness of services’ arrival, the service offloading cost is formulated as a non-convex optimization problem of both power overhead and offloading latency, which satisfies the power limitations and computation performance boundary. Further, an MA prioritized experience replay (PER) DDPG-based power control (MP-DPC) algorithm is proposed, which operates by decentralized training decentralized execution (DTDE) to avoid exponential growth of action space. Especially, PER can accelerate the algorithm convergence and the reward is designed to enhance the DRL training stability. Simulation results verify that our proposed MP-DPC can effectively obtain an optimal service offloading strategy to optimize the service offloading cost in different layers, and significantly outperform other benchmarks with an average service offloading cost reduction of 7.56%.