Hierarchical Meta-Graph Reinforcement Learning for Collaborative GenAI Model Caching and Inference Scheduling

H2-LBM: A Hierarchical Hybrid Deep Reinforcement Learning Framework for L7 Load Balancing and Global Traffic Scheduling in Multi-Cloud LLM Serving

Large Language Model (LLM) inference services are increasingly deployed across multi-cloud infrastructures to support latency-sensitive applications such as conversational AI, intelligent search, and real-time analytics. However, the coexistence of heterogeneous computing resources, geographically distributed data centers, and highly dynamic request patterns poses significant challenges to efficient L7 load balancing and global traffic scheduling. In particular, traditional rule-based or heuristic-driven approaches fail to capture the semantic variability of LLM workloads, including token length diversity and non-linear inference latency, leading to suboptimal resource utilization and degraded tail latency performance. To address these issues, this paper proposes H2-LBM, a hierarchical hybrid deep reinforcement learning framework designed for multi-cloud LLM serving environments. The framework decomposes the scheduling problem into two coordinated levels: a global scheduler based on Proximal Policy Optimization (PPO) for cross-cloud traffic allocation, and a local scheduler based on Double Dueling Deep Q-Network (D3QN) for fine-grained L7 request dispatching. By incorporating semantic-aware state representations and a multi-objective reward function that jointly optimizes latency, throughput, and resource efficiency, H2-LBM enables adaptive and scalable decision-making under dynamic workloads. Experimental results on a multi-cloud Kubernetes testbed show that H2-LBM reduces P99 latency by 21.5%–29.2% compared with strong baselines such as Kubernetes HPA across moderate and burst workloads. Specifically, P99 latency decreases from 360 ms to 255 ms under moderate load and from 650 ms to 510 ms under high load. These results consistently validate the effectiveness of H2-LBM in improving latency stability and system efficiency for large-scale LLM inference services.

Multi-Timescale Latent-Action DRL for Joint Optimization in Edge-Cloud Networks

A two-timescale multi-layer deep reinforcement learning framework with a latent action space (2T-MDRL-LA) to jointly optimize service placement, user association, computational delegation, task offloading, and user transmit power and achieves near-optimal performance compared to branch-and-bound solutions.

Meta-Reinforcement Learning for Computation Offloading and Resource Allocation in MEC-Enabled Immersive Metaverse

MULTI-AGENT REINFORCEMENT LEARNING FOR TASK OFFLOADING AND RESOURCE ALLOCATION IN MEC SYSTEMS

This paper addresses the joint task offloading and resource allocation problem in multi-user MEC systems and proposes a decentralized control framework based on Multi-Agent Reinforcement Learning (MARL), which achieves lower total system cost and faster convergence than the full-local, full-offload, and heuristic baselines.

Graph-MARL-CloudLLM: A Graph-enhanced Multi-Agent Reinforcement Learning Framework for Adaptive Resource Scheduling and Workload Optimization of Large Language Model Inference Services across Multi-Cloud Platforms

The increasing adoption of Large Language Models (LLMs) in enterprise artificial intelligence applications has created significant challenges for inference service management across multi-cloud platforms, where heterogeneous computing resources, dynamic workloads, network fluctuations, and strict quality-of-service (QoS) requirements hinder efficient resource utilization and stable service delivery. This paper aims to develop an intelligent scheduling framework for cross-cloud LLM inference services that can achieve adaptive workload orchestration, low-latency inference, and cost-efficient resource allocation. We propose Graph-MARL-CloudLLM (Graph-enhanced Multi-Agent Reinforcement Learning for Cross-Cloud Large Language Model Inference Optimization), a novel framework that integrates Graph Neural Networks (GNNs) with Multi-Agent Reinforcement Learning (MARL) to model complex cloud resource dependencies and perform autonomous scheduling decisions. Specifically, Graph-MARL-CloudLLM constructs a dynamic resource graph containing cloud nodes, GPU clusters, LLM instances, and inference requests, and employs cooperative agents for cloud selection, model selection, GPU allocation, and request routing. A QoS-aware reinforcement learning strategy is further designed to jointly optimize latency, throughput, cost, and resource utilization. Experimental results on a simulated heterogeneous multi-cloud LLM inference environment demonstrate that Graph-MARL-CloudLLM achieves superior scheduling performance compared with existing approaches. Specifically, the proposed framework reduces P99 latency from 271 ms to 194 ms, improves GPU utilization to 86.9%, and decreases inference cost by 17.7% compared with the MARL-based baseline. These results validate the effectiveness of graph-enhanced state representation and cooperative multi-agent optimization for adaptive LLM inference scheduling.

Graph-Centric Deep Q-Learning for Interference-Aware Resource Allocation in Rsma-Enabled 5G Slicing

The emergence of 5G and 6G advanced ecosystems demands highly adaptive resource management to orchestrate the specialised requirements of eMBB, URLLC, and mMTC network slices. In dense multi-cell environments, capturing complex spatial interdependencies and mitigating dynamic interference is paramount for maintaining Quality of Service (QoS). This paper introduces a robust GNN-DQN framework designed for Rate Splitting Multiple Access (RSMA) based networks. By representing the network topology as a graph, the framework leverages Graph Neural Networks (GNNs) to extract highdimensional spatial features and model inter-cell interference patterns. These insights enable a Deep Q-Network (DQN) agent to perform intelligent resource partitioning and dynamic power splitting of the RSMA common stream. Experimental results demonstrate that the proposed GNN-DQN framework achieves a connectivity success ratio exceeding 90% across all slices, representing an average improvement of over 60% compared to non-graph-based reinforcement learning and supervised baselines. Notably, the framework demonstrates exceptional spectral efficiency, maintaining near-total connectivity while utilising less than 10% of the normalised system bandwidth, a 4× reduction in resource overhead compared to traditional methods. Furthermore, the GNN-driven architecture ensures stable convergence during training, yielding a 1.6× higher system reward score. Our findings validate GNN-DQN as a high-performance, scalable, and resource-efficient paradigm for intelligent orchestration in 5G and 6G networks.