2026· IEEE Open Journal of the Communications Society· Vol 7, pp. 8040-8055· 0 citations· 45 references
Computer Science
TL;DR
A Graph-Embedded En-Route Reinforcement Learning (GEERL) framework that efficiently encodes network topology into compact graph representations by exploiting topological correlations between structure and traffic patterns, enabling fast per-hop decisions without centralized planning overhead.
Abstract
The proliferation of heterogeneous compute-intensive services, such as large language models (LLMs), metaverse, cloud virtual reality (VR), and digital twins, poses unprecedented challenges to computing power-based optical transport networks (OTNs). These emerging applications require fine-grained bandwidth provisioning coupled with intelligent compute node selection, creating a natural contradiction with the static and rigid OTN pipeline architecture. While fine-grain OTN (fgOTN) enables dynamic orchestration through a multi-tier mapping hierarchy: fgOTN timeslots to Optical Data Units (ODUk) to wavelengths, the resulting massive action space and long-latency global state collection severely limit real-time decision-making. This paper proposes a Graph-Embedded En-Route Reinforcement Learning (GEERL) framework that efficiently encodes network topology into compact graph representations by exploiting topological correlations between structure and traffic patterns, enabling fast per-hop decisions without centralized planning overhead. By performing hierarchical per-hop resource allocation, GEERL achieves autonomous network orchestration with significantly reduced latency. Simulation results on the NSFNET and GERMANY50 topologies demonstrate that GEERL reduces average task completion delay by up to 48% and blocking probability by up to 78% compared with centralized Global-DQN planning approaches, while maintaining comparable resource utilization. The proposed en-route decision mechanism significantly reduces the estimated local decision latency compared with centralized schemes, supporting low-latency online orchestration under varying offered-load conditions.
The emergence of 5G and 6G advanced ecosystems demands highly adaptive resource management to orchestrate the specialised requirements of eMBB, URLLC, and mMTC network slices. In dense multi-cell environments, capturing complex spatial interdependencies and mitigating dynamic interference is paramount for maintaining Quality of Service (QoS). This paper introduces a robust GNN-DQN framework designed for Rate Splitting Multiple Access (RSMA) based networks. By representing the network topology as a graph, the framework leverages Graph Neural Networks (GNNs) to extract highdimensional spatial features and model inter-cell interference patterns. These insights enable a Deep Q-Network (DQN) agent to perform intelligent resource partitioning and dynamic power splitting of the RSMA common stream. Experimental results demonstrate that the proposed GNN-DQN framework achieves a connectivity success ratio exceeding 90% across all slices, representing an average improvement of over 60% compared to non-graph-based reinforcement learning and supervised baselines. Notably, the framework demonstrates exceptional spectral efficiency, maintaining near-total connectivity while utilising less than 10% of the normalised system bandwidth, a 4× reduction in resource overhead compared to traditional methods. Furthermore, the GNN-driven architecture ensures stable convergence during training, yielding a 1.6× higher system reward score. Our findings validate GNN-DQN as a high-performance, scalable, and resource-efficient paradigm for intelligent orchestration in 5G and 6G networks.
Aya Kh. Ahmed, Nadia Al-Aboody, Hamed S. Al-Raweshidy· International Conference on...· 0 citations
Routing optimization in cloud-edge collaborative networks faces a fundamental conflict between global strategic planning and local real-time responsiveness, further complicated by structural heterogeneity and stochastic traffic patterns. Traditional protocols lack adaptivity, while existing Deep Reinforcement Learning (DRL) approaches based on Graph Neural Networks (GNN) struggle with limited receptive fields and over-smoothing issues in large-scale topologies. In this paper, we propose HAT-Route, a Transformer-driven hierarchical routing framework supported by the Network Digital Twin (NDT). Our contributions are threefold: 1) We establish a cloud-edge collaborative architecture operating under the Centralized Training and Decentralized Execution paradigm. This architecture balances the trade-off between global optimization and real-time inference. 2) We introduce FlowFormer, a Spatiotemporal Transformer for the NDT. FlowFormer integrates a novel Edge-Conditioned Spatial Attention (EC-SAT) mechanism to capture physical link constraints and distinguish between congestion and Head-of-Line (HOL) blocking. 3) We design HAT-Route, a hierarchical DRL agent that utilizes Graph Transformers for global policy learning in the cloud, coupled with knowledge distillation to deploy lightweight policies at the network edge. Extensive experiments demonstrate that our framework outperforms traditional protocols and GNN-based baselines in terms of QoS optimization, training stability, scalability, and generalization capability on large-scale network topologies.
Bin Dai, Yuntao Wang, Jianhai Zheng· IEEE Transactions on Network...· 0 citations
SAGE (Scene-Adaptive Graph-Enhanced offloading), a task-offloading framework that combines a heterogeneous graph neural network (HeteroGNN) with a dueling double DQN meta-controller and a mixed-integer linear programming (MILP) solver, is proposed, demonstrating strong scene-adaptive decision-making capability.
Lingtao Xue, Xuewen Dong, Xinyu Hu et al.· Electronics· 0 citations
Large Language Models (LLMs) have been increasingly applied to intelligent vehicular systems for tasks such as scene understanding, intent reasoning, and natural language interaction. However, their inference demands exceed onboard processing capabilities, making low-latency on-vehicle inference impractical. Although edge computing can partially offload computation, the prolonged nature of LLM inference often causes execution to exceed the residence time of vehicles within edge coverage areas, leading to frequent service interruption. To address these challenges, we propose a collaborative spatiotemporal resource orchestration architecture for LLM inference in vehicular-edge networks (CoInfer). CoInfer exploits the intrinsic decomposability of LLM inference by modeling each request as a Directed Acyclic Graph (DAG) of interdependent subtasks, which are then scheduled, migrated, and aggregated along the road network to preserve end-to-end inference continuity. To improve latency and resource efficiency, CoInfer integrates multi-agent reinforcement learning for coarse-grained task orchestration with a reactive scheduler for fine-grained resource adaptation, forming a closed-loop service optimization under dynamic resource conditions. The simulation results demonstrate that CoInfer achieves a task success ratio of up to 96.0% and reduces the end-to-end inference latency by 35.7% compared to representative baselines.
Xiwen Liao, Supeng Leng, Ke Zhang et al.· IEEE Transactions on Network...· 0 citations
Deploying Large Language Models (LLMs) over the edge-cloud continuum faces severe stability challenges due to the conflict between stochastic network topology and complex workflow dependencies. Existing schedulers, relying either on computationally prohibitive Graph Neural Networks (GNNs) or topology-agnostic heuristics, fail to reconcile this tension. To bridge these gaps, we propose STEM, a service-level and topology-aware orchestration framework that formulates distributed LLM serving as a workflow-aware routing problem over a monitored service overlay, in which heterogeneous service instances act as specialized experts. At the core of STEM lies the STAR-PPO algorithm, utilizing a lightweight graph-free perception mechanism. By leveraging Squeeze-and-Excitation attention, it extracts critical bottleneck features from raw telemetry with linear complexity, bypassing the scalability limits of message-passing paradigms. To further achieve Pareto-efficient trade-offs, we develop a Dynamic Weight Adaptation (DWA) mechanism that autonomously recalibrates optimization preferences based on entropy-regularized metric drift. Extensive experiments on real-world datasets spanning 2,000 nodes demonstrate that our framework significantly outperforms state-of-the-art baselines. Specifically, STAR-PPO reduces network transmission costs by 96.8% and improves comprehensive inference efficiency by 24.4%, while sustaining robust zero-shot generalization across regions, with average latency within $1.09\times $ of a target-domain-retrained reference under a strict cross-region protocol. Code and data are available at https://github.com/gymorsiback/STARPPO
Yan Gao, Shaoyuan Huang, Yonghui Ye et al.· IEEE Transactions on Cogniti...· 0 citations
Large Language Model-based Multi-Agent Systems (LLM-MAS) have shown exceptional promise for complex tasks, including retrieval-augmented generation and autonomous data analytics. However, their deployment in resource-constrained industrial environments faces critical challenges, such as unpredictable end-to-end latency, low task success rates, and expensive operational costs. Existing orchestration strategies for LLM-MAS mainly rely on static heuristics, implicit LLM-based routing, or reinforcement learning, which suffer from brittleness under workload drift, prohibitive online exploration costs, and inherent governance risks. To address these issues, we propose G-STAR, a general graph-based scheduling framework that formalizes complex MAS pipelines as attributed Directed Acyclic Graphs (DAGs). Specifically, G-STAR decouples its learning pipeline into three critical stages to ensure both system stability and execution efficiency. First, we build a data layer that logs fine-grained signals for agent node (e.g., execution latency, token I/O, model IDs, resource metrics) and task-grounded outcomes (e.g., accuracy, cost, SLA), yielding a trace-driven dataset. Second, we train an offline GNN model to predict optimal dynamic routing decisions, including agent activation, concurrency-constrained Top-K selection, and execution ordering, conditioned on the evolving graph state. Finally, a lightweight dispatcher deploys the pre-trained GNN model for single-forward-pass routing with negligible computational overhead. Extensive experiments on two public benchmarks and our real-world production workload confirm G-STAR's superiority over static and LLM-driven baselines. It maintains competitive task quality while reducing latency variability and consistently improving end-to-end latency. Furthermore, we develop an industry-grade orchestration stack with asynchronous execution, resilient serving, and audit-friendly artifacts, offering a practical solution for optimizing web-scale deployments of complex MAS pipelines.
Jiabao Song, Yunsheng Xia, Beibei Kong et al.· Proceedings of the 32nd ACM...· 0 citations