PRESAGE: Uncertainty-Aware Predictive Elastic Scheduling for Cloud Analytics and GenAI Inference
TL;DR
Presage, an end-to-end framework that reduces forecast CRPS by up to 28% over the best zero-shot TSFM while attaining near-nominal coverage at every quantile the scheduler consumes, and cuts SLO violations by 3– $5\times $ at $\sim \!18\%$ lower resource cost than strong predictive and RL baselines.
Abstract
Cloud data platforms increasingly co-locate hybrid analytical workloads (data-lake/warehouse queries, batch and stream engines) with generative-AI (GenAI) inference services, producing load traces that are strongly bursty, heavy-tailed, and multi-phase. Reactive threshold autoscalers and simple ARIMA/LSTM predictors react poorly to such dynamics, causing service-level-objective (SLO) violations or resource waste. Time-series foundation models (TSFMs) offer strong zero-shot forecasting, but they are not adapted to multivariate, non-stationary cloud load, and, critically, forecasting is usually decoupled from scheduling, so predictive uncertainty never reaches the decision. We present Presage, an end-to-end framework that closes this loop. Presage 1) domain-adapts a pretrained TSFM to multivariate cloud traces through parameter-efficient fine-tuning and continual pretraining with a lightweight cross-channel adapter, enabling few-shot cross-cluster transfer; 2) introduces retrieval-augmented time-series forecasting, conditioning the predictor on the most similar historical windows drawn from a pattern library to capture periodic events and recurring spikes; 3) propagates calibrated probabilistic forecasts (conformalized quantiles) into an uncertainty-aware reinforcement-learning scheduler that chooses a per-service risk level under an explicit cost–SLO constraint, with dedicated prefill/decode provisioning for GenAI; and 4) jointly performs load forecasting and anomaly-precursor detection for robustness. We instantiate Presage on a publicly available TimesFM-1.0-200M backbone and evaluate it on six public traces from Alibaba (a GenAI Stable-Diffusion serving trace), Microsoft Azure (two LLM-inference traces, serverless functions, and VMs), and Google Borg, using a trace-driven simulator, a real vLLM serving harness, and a small 8-GPU Kubernetes testbed. In this evaluation Presage reduces forecast CRPS by up to 28% over the best zero-shot TSFM while attaining near-nominal coverage at every quantile the scheduler consumes, and cuts SLO violations by 3– $5\times $ at $\sim \!18\%$ lower resource cost than strong predictive and RL baselines, with KV-cache out-of-memory events reduced from 27 to 1 for GenAI serving; an adaptive online conformal variant further improves worst-window coverage under drift. Because the coverage guarantees we inherit are marginal and hold only under approximate exchangeability, and because the testbed is small-scale, we state the resulting scope limits explicitly. The code will be released publicly upon acceptance.