Master-Refined MAPPO for Long-Term Joint Resource Scheduling in NOMA-MEC Systems
TL;DR
This study jointly optimizes task offloading and system resource scheduling to minimize the long-term delay–energy cost of NOMA-MEC systems using a master-refined multi-agent proximal policy optimization algorithm.
Abstract
Mobile edge computing (MEC) enables resource-constrained user devices (UDs) to obtain low-latency computing services by offloading computational tasks to the network edge. Non-orthogonal multiple access-enabled mobile edge computing (NOMA-MEC) systems feature asymmetric states across UDs, dynamic task arrivals, and competition for wireless and edge computing resources. Under these conditions, offloading decisions affect device energy consumption, task delay, and edge computing resource allocation, making long-term system optimization difficult. This study jointly optimizes task offloading and system resource scheduling to minimize the long-term delay–energy cost. The problem is formulated as a partially observable Markov decision process (POMDP) and addressed using a master-refined multi-agent proximal policy optimization (MR-MAPPO) algorithm. MR-MAPPO combines continuous action relaxation, master action refinement, and a behavior cloning auxiliary term to learn policies in a hybrid discrete–continuous action space. A marginal congestion delay term is also introduced to capture the impact of newly admitted tasks on existing edge workloads. Simulation results show that MR-MAPPO outperforms the considered baselines, while ablation studies verify the effects of its key components. Under the main experimental setting, MR-MAPPO reduces the system cost by 17.9% and 22.9% relative to standard MAPPO and particle swarm optimization (PSO), respectively.