LMEdge: QoS-Aware LLM Inference Orchestration on Edge Clusters

CELLServe: An SLO-Aware and Cost Efficient LLMs Serving System for Serverless Computing Environments

CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.

TAILOR: Token-Aware Partitioning and Routing for Edge–Cloud Transformer Inference

TAILOR is presented, a token-aware inference framework that jointly optimizes offline partitioning and online routing for autoregressive transformer serving that reduces end-to-end latency and lowers OOM-induced fallback compared with static partitioning baselines, demonstrating robust inference under long-tailed token workloads.

Workflow-Aware Expert Routing for Distributed LLM Serving Over the Edge-Cloud Continuum

Deploying Large Language Models (LLMs) over the edge-cloud continuum faces severe stability challenges due to the conflict between stochastic network topology and complex workflow dependencies. Existing schedulers, relying either on computationally prohibitive Graph Neural Networks (GNNs) or topology-agnostic heuristics, fail to reconcile this tension. To bridge these gaps, we propose STEM, a service-level and topology-aware orchestration framework that formulates distributed LLM serving as a workflow-aware routing problem over a monitored service overlay, in which heterogeneous service instances act as specialized experts. At the core of STEM lies the STAR-PPO algorithm, utilizing a lightweight graph-free perception mechanism. By leveraging Squeeze-and-Excitation attention, it extracts critical bottleneck features from raw telemetry with linear complexity, bypassing the scalability limits of message-passing paradigms. To further achieve Pareto-efficient trade-offs, we develop a Dynamic Weight Adaptation (DWA) mechanism that autonomously recalibrates optimization preferences based on entropy-regularized metric drift. Extensive experiments on real-world datasets spanning 2,000 nodes demonstrate that our framework significantly outperforms state-of-the-art baselines. Specifically, STAR-PPO reduces network transmission costs by 96.8% and improves comprehensive inference efficiency by 24.4%, while sustaining robust zero-shot generalization across regions, with average latency within $1.09\times $ of a target-domain-retrained reference under a strict cross-region protocol. Code and data are available at https://github.com/gymorsiback/STARPPO

Sparsity-Aware Scheduling for Efficient MoE-Based LLM Serving on Edge Servers

A Collaborative Edge Intelligence Framework for SFC Provisioning via Language Models

As Software-Defined Networking (SDN) and Network Function Virtualization (NFV) enabled networks scale in size and complexity, monitoring and managing Service Function Chains (SFCs) under stringent latency and resource constraints becomes increasingly challenging. Although Deep Reinforcement Learning (DRL) is widely applied to SFC provisioning and Virtual Network Function (VNF) placement, enhanced network state monitoring is crucial to capture unexpected network conditions and guide DRL agents toward more adaptive decisions. In this context, Language Models (LMs) enable flexible, natural-language (NL)–based, query-driven network monitoring; however, directly processing complex multi-metric NL queries is computationally expensive and error-prone. This paper proposes an end-to-end (E2E) edge-based query translation pipeline that decomposes multi-metric NL queries into simpler single-metric sub-queries. Query decomposition is performed using a retrieval-augmented language model (RAG-LLM) and compared with a lightweight rule-based decomposition baseline. The resulting sub-queries are translated into Structured Query Language (SQL) using FLAN-T5. A cloud-only baseline, which directly translates NL queries to SQL without decomposition, is also evaluated. The results show that the rule-based edge pipeline achieves the lowest latency, reducing E2E latency by up to 78% compared to RAG-LLM and 18% compared to cloud execution under high workloads. Under increasing arrival rates for the largest workload, the rule-based edge pipeline maintains superior performance over cloud, reducing total E2E latency by 57% at $\lambda = 0.8$ . While RAG-LLM provides greater flexibility for unseen query patterns, both edge-based approaches achieve 100% NL2SQL accuracy with zero decomposition failures, outperforming the cloud-only baseline (95% accuracy).

Performance modeling in FaaS workloads at the edge using machine learning

The Function as a Service (FaaS) paradigm has emerged as a compelling architectural model for both cloud and edge computing environments, enabling the execution of self-contained functions triggered by specific events while abstracting from developers infrastructure management complexities such as load balancing and auto-scaling. In FaaS-enabled clusters, particularly within resource-constrained edge environments, precise resource consumption estimation becomes critical to optimize resource utilization, minimize latency, prevent system overloads, and ensure scalability. This paper addresses performance modeling challenges in FaaS-enabled distributed and decentralized edge computing systems, operating at the granularity level of both nodes and individual functions. We propose a Machine Learning-based framework designed to predict key performance indicators, including CPU utilization, memory, and energy consumption, based on incoming workload patterns, while simultaneously forecasting potential system overload conditions. Moreover, our approach introduces a profiling methodology that characterizes serverless functions according to their resource consumption profiles, thereby enabling accurate prediction of node-level resource demands without requiring detailed knowledge of individual deployed functions. Experimental validation demonstrates that our predictive models achieve 97% accuracy in anticipating node overload scenarios, providing a robust foundation for proactive resource management in edge-based FaaS deployments. Moreover, while our best individual, function-based regression models predict node-level CPU, RAM and power consumption with a Mean Absolute Percentage Error below 9% on average, our experiments highlight the effectiveness of function profiling and cluster-based modeling. On one hand a novel multi-target regressor based on a permutation-invariant neural architecture is proved to generalize effectively across previously unseen workload compositions in the tested scenarios, with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R^2$$\end{document} scores ranging from 0.94 to 0.98, i.e., aligned with the performance of specialized single-target regression models. On the other hand, cluster-level models generalize effectively to previously unseen functions within the same usage class, maintaining prediction errors within practically acceptable ranges and often in the single-digit percentage range.