2026· IEEE Transactions on Cognitive Communications and Networking· Vol 12, pp. 11462-11477· 0 citations· 34 references
Abstract
Deploying Large Language Models (LLMs) over the edge-cloud continuum faces severe stability challenges due to the conflict between stochastic network topology and complex workflow dependencies. Existing schedulers, relying either on computationally prohibitive Graph Neural Networks (GNNs) or topology-agnostic heuristics, fail to reconcile this tension. To bridge these gaps, we propose STEM, a service-level and topology-aware orchestration framework that formulates distributed LLM serving as a workflow-aware routing problem over a monitored service overlay, in which heterogeneous service instances act as specialized experts. At the core of STEM lies the STAR-PPO algorithm, utilizing a lightweight graph-free perception mechanism. By leveraging Squeeze-and-Excitation attention, it extracts critical bottleneck features from raw telemetry with linear complexity, bypassing the scalability limits of message-passing paradigms. To further achieve Pareto-efficient trade-offs, we develop a Dynamic Weight Adaptation (DWA) mechanism that autonomously recalibrates optimization preferences based on entropy-regularized metric drift. Extensive experiments on real-world datasets spanning 2,000 nodes demonstrate that our framework significantly outperforms state-of-the-art baselines. Specifically, STAR-PPO reduces network transmission costs by 96.8% and improves comprehensive inference efficiency by 24.4%, while sustaining robust zero-shot generalization across regions, with average latency within $1.09\times $ of a target-domain-retrained reference under a strict cross-region protocol. Code and data are available at https://github.com/gymorsiback/STARPPO
This paper employs five lightweight machine learning models to predict query-specific latency, accuracy, resource usage, and response size for each model-size-quantization-device combination, and design a lightweight heuristic that approximates the BILP solution.
Reza Farahani, Zoha Azimi, Mario Colosi et al.· 0 citations
Agentic AI applications form an emerging serving workload in which a request creates a workflow: a directed acyclic graph of LLM and tool calls that exposes per-node model choices and optional quality operators such as verifiers. This workload falls between two existing layers. Model-serving engines execute individual calls efficiently but cannot see workflow structure, while agent frameworks fix the workflow but cannot see backend load, so neither jointly chooses each node's model, verifier, and backend under serving-time conditions. We present Dyserve, a workflow-aware serving layer that fills this gap. Dyserve compiles each workflow's per-node model and verifier choices in one integer linear program (ILP) over a heterogeneous backend pool, priced by skill-conditioned offline profiles that transfer across workflows. This couples with hardware entering only through per-model throughput sweeps, and is weighted to concentrate strong models and verification on the nodes whose errors propagate the furthest. Because no single latency-quality preference fits every workload mix, Dyserve pre-solves the program at several pressure levels at admission and shifts a workflow's uncommitted suffix among these strategies under load, keeping the solver off the load-shift path; a failed tool call triggers a one-time residual re-solve that preserves committed work.
Jiayi Qian, Zishen Wan, Hanchen Yang et al.· 0 citations
O-DAG is presented, an end-to-end framework that closes the SAGA--simulation gap and evaluates five scheduling algorithms for a slice scheduling application across various configurations spanning 5K--50K UEs, 2--20 cells, and 2--10 network slices.
Routing optimization in cloud-edge collaborative networks faces a fundamental conflict between global strategic planning and local real-time responsiveness, further complicated by structural heterogeneity and stochastic traffic patterns. Traditional protocols lack adaptivity, while existing Deep Reinforcement Learning (DRL) approaches based on Graph Neural Networks (GNN) struggle with limited receptive fields and over-smoothing issues in large-scale topologies. In this paper, we propose HAT-Route, a Transformer-driven hierarchical routing framework supported by the Network Digital Twin (NDT). Our contributions are threefold: 1) We establish a cloud-edge collaborative architecture operating under the Centralized Training and Decentralized Execution paradigm. This architecture balances the trade-off between global optimization and real-time inference. 2) We introduce FlowFormer, a Spatiotemporal Transformer for the NDT. FlowFormer integrates a novel Edge-Conditioned Spatial Attention (EC-SAT) mechanism to capture physical link constraints and distinguish between congestion and Head-of-Line (HOL) blocking. 3) We design HAT-Route, a hierarchical DRL agent that utilizes Graph Transformers for global policy learning in the cloud, coupled with knowledge distillation to deploy lightweight policies at the network edge. Extensive experiments demonstrate that our framework outperforms traditional protocols and GNN-based baselines in terms of QoS optimization, training stability, scalability, and generalization capability on large-scale network topologies.
Bin Dai, Yuntao Wang, Jianhai Zheng· IEEE Transactions on Network...· 0 citations
The fast development of Edge AI has moved the intelligence towards distributed Edge-Cloud computing infrastructure moving away from centralized cloud in terms of low-latency, privacy-preserving, and context-aware applications. However, the deployment and management of the AI-based workloads in heterogeneous and resourceconstrained environments faces mounting complexity with the need for continuous and autonomous orchestration mechanisms. This paper provides a survey on Edge AI orchestration along the Edge-Cloud continuum based on a closed-loop lifecycle encompassing monitoring, decision making, placement, scaling, migration, and adaptation. We examine the ways in which modern systems sense dynamic environments, reason over multi-objective constraints, and perform runtime control actions to adapt and optimize performance, efficiency, and quality of service. Additionally, we systematically review state-of-the-art approaches starting from classic optimization, to emerging federated intelligence, autonomous agents, and foundation model-driven paradigms. Lastly, we identify key open challenges, including cross-layer monitors, explainable decision making, and continual adaptation under concept drift, outlining future research directions oriented towards fully autonomous and selfoptimizing Edge AI systems.
Dragan Stojanović, N. Stojanović, Aleksandra Stojnev· International Scientific Con...· 0 citations
Edge computing is a promising paradigm for deploying latency-sensitive applications (Apps) as it brings resources closer to end users. Edge Apps often adopt a microservice (MS) architecture, breaking monolithic Apps into lightweight, containerized MSs that can be dynamically and independently deployed. However, managing such Apps involves three key challenges: (i) optimizing the placement of MSs to reduce both response time and resource overhead, (ii) handling MS migration or relocation as users move while minimizing App service disruption (App downtime), and (iii) enabling MS sharing across Apps while ensuring performance guarantees. We formulate this as an optimization problem, named Multi-microservice Application Placement (MAP), prove its NP-hardness, and introduce STEP (State and Topology-aware Edge-MS Placement), a polynomial-time heuristic. STEP distinguishes itself from prior work by: (i) jointly considering stateful and stateless MS characteristics in deployment decisions, (ii) exploiting MS shareability to reduce resource usage, (iii) balancing response latency, App downtime, and resource utilization, and (iv) leveraging multiple versions of the same MS to adapt quality of service to available edge resources. Our results in a small-scale scenario show that STEP achieves near-optimal performance with only 7% higher CPU cost than the optimal solution. Large-scale real-time experiments on a Kubernetes cluster demonstrate that STEP consistently outperforms competing methods, achieving up to 50% lower deployment costs while delivering 50% gain in app quality and saving 15% in radio resources with over 90% request success rates.
Madhura Adeppady, Yenchia Yu, Ali Rahmanian et al.· IEEE Transactions on Network...· 0 citations