Hierarchical Reinforcement Learning-Based Dual-Objective Caching Optimization for Dynamic–Static Metaverse Scenes

Abstract

As Metaverse scenarios grow increasingly complex, users place higher demands on real-time rendering and interactive experiences for static and dynamic resources. Existing research largely focuses on static scene transmission, which is inadequate for joint rendering with dynamic Non-Player Characters (NPC). Moreover, centralized rendering architectures encounter severe latency, bandwidth congestion, and cloud overload under high-concurrency conditions. To address these limitations, we propose a Cloud–Edge–End collaborative architecture for Metaverse scene transmission and rendering. Static subscenes and dynamic NPC resources are compressed and pre-cached on high-performance edge cluster servers, enabling localized rendering and fast response. We further design a Dual-objective Trajectory Prediction (DTP) method that integrates user trajectories with NPC behavioral features, employing Transformer-based sequence modeling and TimeSeriesKMeans clustering to jointly predict future static scenes and NPC states. Building on these predictions, we develop a Metaverse Hierarchical Reinforcement Learning (MetaHRL) strategy: the upper level dynamically adjusts the caching ratio between static and dynamic resources using Soft Actor–Critic (SAC), while the lower level performs replacement operations using Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to achieve optimized dual-resource caching. Experimental evaluation demonstrates that our approach increases rendering success rate by 17% and reduces cache replacement frequency by 38.3%, while improving cache hit rate and system stability, showing strong practicality, robustness, and scalability for dynamic–static rendering scheduling in the Metaverse.