Multi-Timescale Latent-Action DRL for Joint Optimization in Edge-Cloud Networks

MULTI-AGENT REINFORCEMENT LEARNING FOR TASK OFFLOADING AND RESOURCE ALLOCATION IN MEC SYSTEMS

This paper addresses the joint task offloading and resource allocation problem in multi-user MEC systems and proposes a decentralized control framework based on Multi-Agent Reinforcement Learning (MARL), which achieves lower total system cost and faster convergence than the full-local, full-offload, and heuristic baselines.

Hierarchical Meta-Graph Reinforcement Learning for Collaborative GenAI Model Caching and Inference Scheduling

Enabling collaborative generative AI (GenAI) inference at the network edge is challenging due to limited caching capacity, heterogeneous computing resources, and highly dynamic, latency-sensitive service demands. In this paper, we investigate the joint optimization of GenAI model caching, inference offloading, and resource allocation in a collaborative cloud–edge–end architecture. To address the strong coupling between long-term caching decisions and short-term scheduling dynamics, we propose a Hierarchical Meta-Graph Reinforcement Learning framework, termed HMGRL. Specifically, a heat-greedy model caching strategy is developed to capture time-varying model popularity and to reduce switching overhead on a slow timescale, while a graph-enhanced dueling deep reinforcement learning algorithm with prioritized experience replay enables topology-aware collaborative inference offloading and resource allocation on a fast timescale. Extensive simulations demonstrate that HMGRL consistently outperforms representative baselines in terms of system utility, cache and computing-resource utilization, convergence stability, and performance robustness. These results validate the effectiveness of the proposed hierarchical learning framework for practical GenAI applications at the network edge.

GMM-TDQN: Two-Stage Multi-Objective Reinforcement Learning for Large-Scale Edge Server Deployment

H2-LBM: A Hierarchical Hybrid Deep Reinforcement Learning Framework for L7 Load Balancing and Global Traffic Scheduling in Multi-Cloud LLM Serving

Large Language Model (LLM) inference services are increasingly deployed across multi-cloud infrastructures to support latency-sensitive applications such as conversational AI, intelligent search, and real-time analytics. However, the coexistence of heterogeneous computing resources, geographically distributed data centers, and highly dynamic request patterns poses significant challenges to efficient L7 load balancing and global traffic scheduling. In particular, traditional rule-based or heuristic-driven approaches fail to capture the semantic variability of LLM workloads, including token length diversity and non-linear inference latency, leading to suboptimal resource utilization and degraded tail latency performance. To address these issues, this paper proposes H2-LBM, a hierarchical hybrid deep reinforcement learning framework designed for multi-cloud LLM serving environments. The framework decomposes the scheduling problem into two coordinated levels: a global scheduler based on Proximal Policy Optimization (PPO) for cross-cloud traffic allocation, and a local scheduler based on Double Dueling Deep Q-Network (D3QN) for fine-grained L7 request dispatching. By incorporating semantic-aware state representations and a multi-objective reward function that jointly optimizes latency, throughput, and resource efficiency, H2-LBM enables adaptive and scalable decision-making under dynamic workloads. Experimental results on a multi-cloud Kubernetes testbed show that H2-LBM reduces P99 latency by 21.5%–29.2% compared with strong baselines such as Kubernetes HPA across moderate and burst workloads. Specifically, P99 latency decreases from 360 ms to 255 ms under moderate load and from 650 ms to 510 ms under high load. These results consistently validate the effectiveness of H2-LBM in improving latency stability and system efficiency for large-scale LLM inference services.

Master-Refined MAPPO for Long-Term Joint Resource Scheduling in NOMA-MEC Systems

Mobile edge computing (MEC) enables resource-constrained user devices (UDs) to obtain low-latency computing services by offloading computational tasks to the network edge. Non-orthogonal multiple access-enabled mobile edge computing (NOMA-MEC) systems feature asymmetric states across UDs, dynamic task arrivals, and competition for wireless and edge computing resources. Under these conditions, offloading decisions affect device energy consumption, task delay, and edge computing resource allocation, making long-term system optimization difficult. This study jointly optimizes task offloading and system resource scheduling to minimize the long-term delay–energy cost. The problem is formulated as a partially observable Markov decision process (POMDP) and addressed using a master-refined multi-agent proximal policy optimization (MR-MAPPO) algorithm. MR-MAPPO combines continuous action relaxation, master action refinement, and a behavior cloning auxiliary term to learn policies in a hybrid discrete–continuous action space. A marginal congestion delay term is also introduced to capture the impact of newly admitted tasks on existing edge workloads. Simulation results show that MR-MAPPO outperforms the considered baselines, while ablation studies verify the effects of its key components. Under the main experimental setting, MR-MAPPO reduces the system cost by 17.9% and 22.9% relative to standard MAPPO and particle swarm optimization (PSO), respectively.

Deep reinforcement learning for cost-efficient resource management in hybrid cloud environments

Sensitivity and ablation studies confirm stable learning and controllable latency-cost trade-offs, demonstrating that lightweight RL can effectively deliver cost-efficient, adaptive autoscaling in hybrid cloud environments.