Queue‐Aware Learning‐Based Online Path and Workload Optimization for In‐Network Computing

Abstract

Recently, in‐network computing has been proposed as a new paradigm to integrate computation and communication at the network edge, enabling data to be processed while being transmitted toward its destination. To achieve this, the problem of joint optimization of computation path selection and workload allocation has gained extensive research interest, and many efficient algorithms have been proposed. However, existing studies assume fixed or static latency when selecting computation paths, and thus fail to capture the time‐varying and queueing‐induced latency experienced by tasks arriving dynamically in online environments. To address the above limitations, this paper investigates the online scheduling problem of time‐dependent path selection and workload allocation for in‐network computing. We formulate the problem as a mixed‐integer programming model that jointly minimizes server operating cost and end‐to‐end latency, and prove that it is NP‐hard. To enable efficient online decision‐making, we further transform the problem into a Markov decision process. Based on this formulation, we propose a reinforcement learning–based scheduling algorithm using proximal policy optimization (PPO), which incorporates an LSTM‐based temporal encoder to capture historical deployment information and dynamically evolving network states. This design allows the algorithm to make adaptive, queueing‐aware scheduling decisions in dynamic network environments. Extensive simulations on real networks demonstrate that the proposed algorithm is robust and outperforms state‐of‐the‐art baseline methods in terms of the joint cost–latency performance metric.