Joint Service Caching and Resource Allocation in DT-Empowered Cloud-Edge Networks via MARL With Hierarchical Knowledge Transfer

Abstract

Through bypassing the long-distance transmission of cloud services, Mobile Edge Computing (MEC) reduces response delay and ensures Quality-of-Service (QoS). In resource-constrained MEC, due to the strong coupling between service provisioning and task execution, reasonable service caching and resource allocation face many challenges on 1) coordinating the limited storage resources to improve cache hit rate, 2) allocating limited computing resources under delay constraints, and 3) sample collection and model training in dynamic environments. To address these important challenges, we propose CAMART, a novel service Caching and resource Allocation framework via Multi-Agent Reinforcement learning (MARL) with hierarchical knowledge Transfer in Digital Twin (DT) empowered Cloud-Edge Networks (DTCEN). Specifically, we first construct a new DTCEN to realize the mapping from physical to virtual networks. Next, we decouple the joint optimization of service caching and resource allocation into two sub-problems. For the service caching sub-problem, we design an improved MARL-based method to capture global information via a shared-feature extraction network and optimize caching and offloading decisions via a dual-head feature processing network. For the resource allocation sub-problem, we convert it into 0-1 integer programming and design an improved branch-and-bound-based method to reduce computational complexity while guaranteeing a high-quality solution set. Finally, we develop an original DT-driven hierarchical knowledge transfer mechanism to realize cross-scenario knowledge reuse and convergence acceleration. Using real-world datasets, extensive experiments are conducted to validate the superiority of the proposed CAMART. Compared to the state-of-the-art methods, CAMART achieves higher rewards, task completion rate, and cache hit rate in different scenarios.