Deep Reinforcement Learning-Based Task Scheduling With Queue Dynamics for Edge Computing Load Balance

Abstract

As computing demands continue to grow, a single server is no longer sufficient to meet user requirements, leading to increasing interest in multiserver collaborative edge computing. However, load imbalance is a prevalent issue in multiserver edge computing systems, resulting in inefficient resource utilization and degraded service quality. To address this issue, a multiserver collaborative edge computing architecture is established, and a joint optimization problem is formulated to minimize task latency and energy consumption under latency constraints. Considering the dynamic nature of task arrivals and queue evolution, the problem is further modeled as a Markov Decision Process (MDP). To characterize more accurately the dynamic evolution of computation queue states in the MDP during task transmission, an arrival order-based queue state (AOBQS) model is introduced to capture the impact of transmission delay on task execution order. Furthermore, as transmission delay alters the task execution order in the computation queue and thus invalidates the system’s Markov property, the task waiting time and a virtual queue are introduced to reconstruct the queue state. Based on the reconstructed state representation, a queue-aware twin-delayed deep deterministic policy gradient (QATD3) algorithm is developed to solve the task scheduling and resource allocation problem, thereby achieving load balancing in multiserver collaborative edge computing systems. Extensive simulation results demonstrate that the proposed method effectively achieves joint optimization of task latency and energy consumption, significantly improving overall system performance. Compared with baseline algorithms, the proposed QATD3 reduces average task delay by 24.53%, reduces normalized energy consumption by 16.06%, and improves average reward by 5.27%.