Recently, in‐network computing has been proposed as a new paradigm to integrate computation and communication at the network edge, enabling data to be processed while being transmitted toward its destination. To achieve this, the problem of joint optimization of computation path selection and workload allocation has gained extensive research interest, and many efficient algorithms have been proposed. However, existing studies assume fixed or static latency when selecting computation paths, and thus fail to capture the time‐varying and queueing‐induced latency experienced by tasks arriving dynamically in online environments. To address the above limitations, this paper investigates the online scheduling problem of time‐dependent path selection and workload allocation for in‐network computing. We formulate the problem as a mixed‐integer programming model that jointly minimizes server operating cost and end‐to‐end latency, and prove that it is NP‐hard. To enable efficient online decision‐making, we further transform the problem into a Markov decision process. Based on this formulation, we propose a reinforcement learning–based scheduling algorithm using proximal policy optimization (PPO), which incorporates an LSTM‐based temporal encoder to capture historical deployment information and dynamically evolving network states. This design allows the algorithm to make adaptive, queueing‐aware scheduling decisions in dynamic network environments. Extensive simulations on real networks demonstrate that the proposed algorithm is robust and outperforms state‐of‐the‐art baseline methods in terms of the joint cost–latency performance metric.
A task scheduling method using the Deep Q-Network to determine the computation node for the computation task and a dynamic congestion-aware mechanism to determine a low-cost routing path is proposed, which gradually obtains an effective task scheduling scheme through multiple rounds of alternating iterations.
As computing demands continue to grow, a single server is no longer sufficient to meet user requirements, leading to increasing interest in multiserver collaborative edge computing. However, load imbalance is a prevalent issue in multiserver edge computing systems, resulting in inefficient resource utilization and degraded service quality. To address this issue, a multiserver collaborative edge computing architecture is established, and a joint optimization problem is formulated to minimize task latency and energy consumption under latency constraints. Considering the dynamic nature of task arrivals and queue evolution, the problem is further modeled as a Markov Decision Process (MDP). To characterize more accurately the dynamic evolution of computation queue states in the MDP during task transmission, an arrival order-based queue state (AOBQS) model is introduced to capture the impact of transmission delay on task execution order. Furthermore, as transmission delay alters the task execution order in the computation queue and thus invalidates the system’s Markov property, the task waiting time and a virtual queue are introduced to reconstruct the queue state. Based on the reconstructed state representation, a queue-aware twin-delayed deep deterministic policy gradient (QATD3) algorithm is developed to solve the task scheduling and resource allocation problem, thereby achieving load balancing in multiserver collaborative edge computing systems. Extensive simulation results demonstrate that the proposed method effectively achieves joint optimization of task latency and energy consumption, significantly improving overall system performance. Compared with baseline algorithms, the proposed QATD3 reduces average task delay by 24.53%, reduces normalized energy consumption by 16.06%, and improves average reward by 5.27%.
Jingzhe Wang, Si-yu Lin, Qingqing Pan et al.· IEEE Internet of Things Jour...· 0 citations
In edge–cloud collaborative computing, efficient scheduling of concurrent task chains is essential for reducing end‐to‐end latency. However, heterogeneous resources, dynamic link contention, and coupled routing‐computation decisions make it difficult to improve system efficiency while reducing communication conflicts. To address these challenges, we formulate concurrent heterogeneous task‐chain scheduling under partial observability as a multi‐agent partially observable Markov game and propose a MARL‐based scheduling method. Each task chain is represented by a mobile agent that makes routing, computation, data‐access, and waiting decisions based on local observations. Building upon a multi‐agent proximal policy optimization (MAPPO) architecture, we develop DC‐MAPPO, which introduces a Directional Clamp mechanism to improve policy update stability under high‐concurrency conditions. In addition, a dynamic action masking strategy is designed to ensure decision feasibility and reduce invalid exploration. Experimental results across multiple network topologies show that the proposed method outperforms the compared baselines in most tested settings. Under high‐load conditions, DC‐MAPPO reduces system makespan by 10.7%–17.1% and reduces the link reservation failure rate by 8.4–13.2 percentage points compared with MAPPO.
Xinyi Li, Chao Wang, Jiakai Liang et al.· Concurrency and Computation· 0 citations
Mobile edge computing (MEC) has emerged as a promising paradigm to support latency-sensitive applications by deploying computing and storage resources closer to end users. Service caching at the network edge is an effective approach to further reduce response latency and improve quality of service (QoS). However, the increasing scale and dynamic nature of user requests pose significant challenges to efficient service caching and updating under limited edge resources and cost budgets. In this article, we investigate the joint optimization problem of service caching and dynamic updating in a cloud–edge–end collaborative architecture, with the objective of minimizing the average service access latency of users. We first construct a comprehensive system model that captures communication latency, service response delay, and the costs associated with service caching and updates. The problem is formulated as a nonlinear integer programming problem and addressed via a two-stage solution framework. Specifically, in the initial caching stage, we employ Lagrangian relaxation to reduce the high-dimensional knapsack problem and design an efficient algorithm integrated with Tabu Search. In the dynamic updating stage, we develop a Monte Carlo tree search (MCTS)-based algorithm enhanced by service popularity awareness, along with a novel service replacement strategy that considers user preference and service reliability. Extensive experiments under various system scales demonstrate that the proposed approach significantly outperforms state-of-the-art methods in terms of average service latency, cost budget satisfaction, and update efficiency.
Tianyang Zheng, Pengfei Yang, Kang Shen et al.· IEEE Internet of Things Jour...· 0 citations
Virtualization in 5G and beyond networks enables the creation of virtual networks (i.e., network slices) tailored to the needs of different applications. To maximize revenue under limited infrastructure resources, InPs must decide in real time whether to admit incoming slice requests (SRs) based on their resource demands and offered values, while accounting for the opportunity cost of consuming scarce resources. To address this challenge, we introduce Online Pricing-based Slice Admission Control and Resource Allocation (OPA) framework. This framework dynamically assigns pseudo-prices to resources that capture long-term scarcity and anticipated inter-temporal opportunity costs. The short-term admission and resource allocation decisions for each SR are then guided by these prices. Additionally, we design an exponential pricing strategy that guarantees bounded worst-case performance. To improve practical performance, we further develop a data-driven exponential pricing approach that learns from historical data. Evaluations on a real-world network topology show that it improves mean revenue by 32.2% and 26.7% over state-of-the-art DRL and optimization-based approaches, respectively, while reducing computational cost by an order of magnitude relative to the latter.
Muhammad Sulaiman, Bo Sun, M. A. Salahuddin et al.· 0 citations
An adaptive Beta-policy and delayed-update multi-agent soft actor-critic method, abbreviated as ABDMASAC, which uses a Beta policy to model bounded actions and achieves a better overall trade-off than the selected MASAC-backbone and on-policy MARL baselines under the considered simulation settings.
Zheng Yao, Jie Liu, Changjun Deng et al.· Computers, Materials & C...· 0 citations