DDPG-Attention-Based Resource Allocation and Trajectory Optimization in Hierarchical MEC

Abstract

Multi-access Edge Computing (MEC) can effectively process Internet of Things (IoT) data by transferring computing intensive tasks to edge servers, and has become an effective mechanism to meet the growing demand for computing. The flexible Uncrewed Aerial Vehicle (UAV) and High-Altitude Platform (HAP) with powerful resources working together can significantly improve the efficiency of edge computing system. This paper investigates the resource allocation and trajectory optimization problems in HAP-UAV-MEC system with a Non-Orthogonal Multiple Access (NOMA) communication scenario. By utilizing Wireless Power Transfer (WPT) technology to provide energy support for UAV, we jointly optimize UAV trajectories, resource allocation, and offloading decisions to minimize the energy cost of IoT devices and the energy cost of UAV. This problem is described as a multi-stage Mixed Integer Nonlinear Programming (MINLP) problem. A Deep Deterministic Policy Gradient (DDPG)-Attention-based Resource Allocation and Trajectory Optimization (DART) algorithm combining Deep Reinforcement Learning (DRL) and Lyapunov optimization techniques is proposed to address this issue. DART algorithm utilizes the Lyapunov technique to transform the multi-stage MINLP problem into a deterministic optimization problem, and decomposes the original problem into four parallel subproblems. Through DDPG-attention algorithm based on reinforcement learning and deep learning attention mechanisms, we solve the problems of trajectory optimization and offloading decision. Meanwhile, for remaining subproblems related to resource allocation, convex optimization is used to solve them. The experimental results verify that the DART algorithm can significantly reduce the total cost while ensuring system stability and performance.