E2E-DRTO: a micro end-to-end routing and transmission collaborative optimization method for large scale networks

Abstract

With the rapid development of large-scale networks such as data center networks and wide area networks, how to achieve efficient traffic scheduling under complex topologies and dynamic traffic demands has become a key issue. Traditional methods typically separate routing selection from transmission rate control, or rely on reinforcement learning for joint optimization. However, the former is difficult to achieve global optimality, while the latter suffers from unstable training and low convergence efficiency. This paper proposes E2E-DRTO, a differentiable end-to-end framework for jointly optimizing multi-path routing and transmission rates in large-scale networks. Unlike conventional soft-routing schemes that only relax path selection, the proposed method couples differentiable path splitting with rate control under a unified congestion-aware objective. To improve scalability, each source-destination pair is restricted to a small candidate-path set, which converts the original combinatorial routing problem into a compact differentiable micro-decision space. In addition, a smooth link-delay proxy is derived from a low-order expansion of the queueing-delay function, and its approximation error in the sub-saturation regime is explicitly characterized. Experiments on standardized real and synthetic topologies, averaged over 10 independent runs, show that E2E-DRTO consistently outperforms SP, ECMP-like, soft-routing, RL-based, and GNN-based baselines in average delay and maximum link utilization, while maintaining high demand satisfaction. We further report computational overhead, convergence behavior, and ablation studies on the number of candidate paths and hyperparameter settings, demonstrating that the proposed framework is both effective and practically scalable.