Graph-Embedded En-Route Reinforcement Learning for Fine-Granularity Resource Orchestration in Computing Power Optical Networks

Abstract

The proliferation of heterogeneous compute-intensive services, such as large language models (LLMs), metaverse, cloud virtual reality (VR), and digital twins, poses unprecedented challenges to computing power-based optical transport networks (OTNs). These emerging applications require fine-grained bandwidth provisioning coupled with intelligent compute node selection, creating a natural contradiction with the static and rigid OTN pipeline architecture. While fine-grain OTN (fgOTN) enables dynamic orchestration through a multi-tier mapping hierarchy: fgOTN timeslots to Optical Data Units (ODUk) to wavelengths, the resulting massive action space and long-latency global state collection severely limit real-time decision-making. This paper proposes a Graph-Embedded En-Route Reinforcement Learning (GEERL) framework that efficiently encodes network topology into compact graph representations by exploiting topological correlations between structure and traffic patterns, enabling fast per-hop decisions without centralized planning overhead. By performing hierarchical per-hop resource allocation, GEERL achieves autonomous network orchestration with significantly reduced latency. Simulation results on the NSFNET and GERMANY50 topologies demonstrate that GEERL reduces average task completion delay by up to 48% and blocking probability by up to 78% compared with centralized Global-DQN planning approaches, while maintaining comparable resource utilization. The proposed en-route decision mechanism significantly reduces the estimated local decision latency compared with centralized schemes, supporting low-latency online orchestration under varying offered-load conditions.