Adaptive Dual-Layer DRL-Based AI Model Placement, Task Scheduling, and Resource Allocation for Collaborative Edge Inference and Training

Abstract

In edge intelligence networks, AI model inference and training tasks coexist and complement each other. Inference tasks demand low latency and diversity to support real-time services, whereas training tasks are computationally intensive and rely on distributed user devices (UDs) data for continuous model enhancement.However, existing research often consider inference and training tasks separately, overlooking their joint optimization and the inherently distributed nature of training data. In this paper, we investigate an edge intelligence network that simultaneously supports multiple heterogeneous AI services. We jointly optimize AI model placement, scheduling of inference and training tasks, as well as resource allocation, including transmit power of UDs, transmission rate, and computing resources of edge servers (ESs). To tackle the high complexity of this joint optimization problem, we design an adaptive dual-layer deep reinforcement learning (DRL) framework based on Twin Delayed Deep Deterministic Policy Gradient (TD3), with an adaptive optimization timescale adjustment mechanism. The upper-layer policy network adaptively decides AI model placement, while the lower-layer policy network schedules inference and training tasks. The two policy networks operate at different update frequencies, and the AI model placement strategy decided by the upper-layer policy network is taken as part of the state input for the lower-layer policy network. In addition, three embedded numerical subroutines are developed to solve resource allocation sub-problems efficiently. We conducted extensive comparative experiments, and the results show that the proposed algorithm converges faster and reduces the total system cost by 11–32% compared with five other approaches.