Jul 2026· Journal of Computer Science and Technology Studies· Vol 8, pp. 238-243· 0 citations· 9 references
TL;DR
The result is an interpretable, stability-analyzed, auto-tuned routing controller competitive with or superior to non- stationary bandits at lower cost.
Abstract
Serving a stream of requests across large language models (LLMs) of differing cost and quality is an online allocation problem, usually framed as multi-armed bandits. We frame it as feedback control: a direct-acting Proportional–Integral–Derivative (PID) controller whose setpoint is the running fleet- average performance and whose bounded output adjusts each model’s allocation share, with requests routed by weighted sampling over the allocation vector. This work contributes (i) a stability analysis ofthe closed loop — bounded-input bounded-output behaviour by anti-windup, exponential convergence of the performance estimates via a Lyapunov function, and a persistent-excitation condition guaranteeing recoverability after a regime change; (ii) a closed-form, analysis-grounded automatic tuning rule requiring no per-dataset search; and (iii) an honest head-to-head against static, round-robin, random, epsilon-greedy, UCB1, Thompson sampling, and the non-stationary bandits Sliding-Window UCB and Discounted UCB, on GSM8K with a checkable exact-match reward, reporting inference cost and request latency alongside quality. Under transient drift the auto-tuned controller is statistically tied with the best non-stationary bandit at lower cost; under a persistent regime shift it significantly outperforms both (p < 0.03). We further show the integral term helps only under a persistent shift — a proportional controller suffices for transient drift — and evaluate robustness to noisy rewards. The result is an interpretable, stability-analyzed, auto-tuned routing controller competitive with or superior to non- stationary bandits at lower cost.
A multi-model language service must route each request while preserving workload-level budgets for compute, latency, memory, or monetary cost. Two features make this problem materially harder than static model selection. Prompt representations are high dimensional, so only a small subset of embedding directions may predict the incremental value of a model, and both the request mix and the model frontier drift after launches, fine-tunes, quantization changes, and system updates. We formulate nonstationary sparse contextual routing with multiple knapsack constraints and an optional shadow-audit stream that evaluates a small fraction of prompts on several models. We propose Drift-Aware Sparse Routing (DRS). The policy estimates reward and resource use from a rolling audit window, routes using pessimistic reward and optimistic cost estimates, updates resource shadow prices online, and applies a hard meter before commitment. The analysis separates control from statistics. On any event with uniform prediction radii $\{\beta_t\}$, regret against a paced dynamic fluid benchmark is bounded by the sum of the radii, a capacity-buffer term, and an $O(\sqrt{T})$ pacing term. Under a sparse linear model and bounded drift $V_T$, rolling estimation gives \[ \widetilde O\left( T\sqrt{\frac{s}{\rho W}}+WV_T+\sqrt{T} \right), \] where $s$ is sparsity, $\rho$ is the audit rate, and $W$ is the window length. Optimizing $W$ yields the usual stationary $O(\sqrt{sT/\rho})$ rate when $V_T=0$ and a $O(T^{2/3}(s/\rho)^{1/3}V_T^{1/3})$ adaptation term under drift.
The rapid growth of large-scale AI workloads in data centers has placed increasing pressure on power grids in recent years. Since power systems must continuously balance supply and demand, there is growing interests in leveraging data-center workload flexibility as a grid service. We propose a contextual restless multi-armed bandit (CRMAB) framework in which a grid operator requests load reductions without observing internal job-scheduling decisions. Under index-ability guarantee, each data center or physical machine is modeled as a Markov decision process (MDP) over a cyclic virtual-machine (VM) job queue, with unknown rewards and transition dynamics learned online using Thompson sampling and Whittle-index policies. To improve learning under sparse and noisy observations, the framework augments an adaptive Thompson--Whittle (TW) policy with domain-informed transition priors and gated prior mixing. In baseline experiments, the best adaptive refined variant achieves 91.4\% of the oracle reward after 100 rounds and 96.8\% after 1,000 rounds. Across a 16-setting stress test spanning different state-space sizes and levels of contextual noise, the best refined variant consistently outperforms the original TW policy with high confidence while remaining competitive with EXP4. A graph-based prior further incorporates data-center hardware constraints, including computing-resource limits. Overall, the results demonstrate the economic potential of data-center flexibility as a grid service and highlight the importance of high-quality, open-source AI workload traces for developing and evaluating such services.
Zixi Chen, Yifu Ding, Ruicheng Ao et al.· 0 citations
This paper studies weighted decoding-delay minimization for multiple RLNC-coded message streams that compete for finite processing capacity at a destination. Packet arrivals are exogenous, while the scheduler only determines the processing order of packets already available at the destination. A trace-conditioned offline scheduling formulation shows that a batch-release subclass is strongly NP-hard even with a single processing unit. Message-Aware Innovation-Deficit Scheduling (MAIDS) is then developed to prioritize each serviceable message according to its weight and remaining decoding deficit. For a single processing unit, MAIDS is shown to be exactly optimal under nonblocking progressive arrivals with equal weights and under common activation with arbitrary positive weights, while the unrestricted weighted online problem admits no universal deterministic $O(1)$ competitive ratio. Simulation results on streaming and batch benchmarks show that MAIDS consistently reduces weighted decoding delay relative to the tested baselines, remains close to the offline optimum on average, and recovers the predicted exact performance boundaries.
Mobile edge computing (MEC) must serve rapidly growing populations of latency-critical and energy-constrained devices, yet distributed offloading faces two coupled problems: learned multi-agent policies depend on the arbitrary numerical ordering of edge servers, which wastes training samples and treats physically equivalent configurations inconsistently, while short-horizon cost minimization overloads attractive servers and destabilizes their queues. This paper presents SEAL-MAC (Symmetry-Equivariant Lyapunov Multi-Agent Actor–Critic), a distributed learning framework that addresses both problems jointly. First, a symmetric resource-set actor with a mirror consistency regularizer enforces server relabeling equivariance of each user’s policy and invariance of its value and Lyapunov critics. Second, a load-symmetric Lyapunov–potential shaping mechanism augments drift-plus-penalty rewards with normalized load-balance signals, coupling queue stability, fairness, and strategic alignment. The shaped interaction is analyzed as a Lyapunov-shaped Markov potential game: exact under orthogonal congestion-separable conditions, and a Markov α-potential game under heterogeneity or interference, yielding conditional finite-time (ϵ+α)-Nash convergence and mean-square queue stability. Each device learns from local observations and O(M) queue broadcasts without exchanging gradients or policies. In simulations with up to 200 users, SEAL-MAC reduces average delay by 9.0%, 95th-percentile delay by 11.8%, energy consumption by 10.6%, and the deadline-violation rate from 3.1% to 1.8% relative to the strongest Lyapunov baseline, halves the empirical one-step deviation gain of an identically shaped Ly-PPO agent (0.048 versus 0.098), and raises the Jain fairness index from 0.88 to 0.94.
Elastic compute platforms must provision enough replicas to absorb bursty arrivals while avoiding persistent over-reservation. This paper develops DR-MPC-Elastic, a distributionally robust multi-timescale controller for microservice autoscaling. The method replaces a fixed safety margin with a data-dependent ambiguity radius estimated from the tail of recent forecast residuals. It combines an arrival-service workload model, a Wasserstein distributional uncertainty set, a CVaR tail-risk surrogate, and a mirror-descent projection that produces integer replica decisions with bounded actuation. The empirical case study uses a public Alibaba microservice trace containing 44,903 cleaned records, 390 thirty-second control windows, 381 service identifiers, and 13,060 container identifiers. Compared with a reactive HPA-like policy, DR-MPC-Elastic materially reduces SLA-risk slots while preserving a transparent cost-risk trade-off. The contribution is theoretical, auditable, and implementation-oriented: autoscaling is treated as risk-calibrated online optimization rather than as an opaque reinforcement-learning policy.
Yizhou Chen· Journal of Computational Met...· 0 citations
Multi-agent applications increasingly rely on shared large language model backends in the public cloud, where bursty workloads cause requests from different agents to contend for the same LLM instances, leading to long queues, memory imbalance, and severe tail-latency inflation. Existing approaches typically prioritize requests using coarse workflow positions or static execution heuristics, which fail to adapt to short-term overload dynamics. We present FlowGuard, a workflow-aware overload controller for multi-agent LLM serving. Its key insight is that under sustained overload, GPU cycles spent on requests whose execution service-level-objectives (SLOs) are already violated are wasted. FlowGuard continuously recomputes per-request slack and prioritizes requests with the greatest remaining time before their deadlines, thereby maximizing on-time completions. In addition, a resource-aware dispatcher jointly accounts for KV-cache memory pressure and in-flight queue depth to reduce preemption across shared instances. Evaluated on a deliberately over-subscribed two-GPU backend, where all policies exhibit high absolute miss rates (i.e., the percentage of workflows that miss their deadlines), FlowGuard reduces the miss rate by 14–28% points over workflow-oblivious and static-priority baselines under BurstGPT-driven load, and by 34–38% points under co-located mixed-agent workloads.
Ali Zafar Sadiq, Hai-Ying Shen· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
