TL;DR
CAPA is presented, a confidence-aware predictive autoscaler that couples a lightweight rolling ridge forecaster with an online one-sided residual-quantile bound and an SLO-debt feedback signal and reduces the estimated SLO violation rate relative to a reactive HPA-style baseline.
Large language model (LLM) inference deployed on Kubernetes faces an autoscaling challenge that conventional web services do not: new serving replicas take two to ten minutes to start because multi-gigabyte model weights must be loaded, which makes purely reactive scaling structurally late. We ask a sharp question: amo...
Presage, an end-to-end framework that reduces forecast CRPS by up to 28% over the best zero-shot TSFM while attaining near-nominal coverage at every quantile the scheduler consumes, and cuts SLO violations by 3– $5\times $ at $\sim \!18\%$ lower resource cost than strong predictive and RL baselines.
HARP is presented, an adaptive hybrid framework that treats the choice of control philosophy as an online decision rather than a design-time commitment and cuts SLO violations by 26.8–92.4% on three real production traces driven by a measured CPU inference profile.
SLO-Scaler is proposed, an uncertainty-aware autoscaling framework that predicts short-horizon request rates, tail latency, and SLO violation probability using a Bayesian LSTM model, and is evaluated on the DeathStarBench Social Network benchmark deployed on Kubernetes under periodic, bursty, and long-tail traffic patt...
A reinforcement learning (RL) recommender VERA is presented that formulates vertical memory scaling as a Markov Decision Process and trains an agent on 3353 real Prometheus traces and demonstrates that an observation-driven RL recommender could outperform retrospective heuristics for dynamic memory scaling.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
