MorphCloud-LLM: Elastic Spot-Instance-Aware LLM Serving with Transparent Preemption Recovery and Speculative Decoding Continuity

Abstract

Serving large language models (LLMs) on cloud spot and preemptible instances reduces costs by 60 to 90 percent compared to on-demand pricing, but unpredictable instance preemptions cause request failures, KV-cache state loss, and degraded user experience. We present MorphCloud-LLM, an elastic LLM serving system designed to achieve the reliability properties of on-demand serving at spot-instance pricing. MorphCloud-LLM integrates three synergistic components: (1) an asynchronous incremental KV-cache checkpointing engine that streams only delta state to disaggregated persistent storage with less than 3% throughput overhead, enabling sub-second KV-cache delta streaming and reconstruction for KV-cache sizes up to 32 GB on replacement instances (total end-to-end migration latency: 1390 ms); (2) a gradient-boosted preemption prediction model trained on spot market telemetry that achieves 89% recall at a 30-s prediction horizon, providing sufficient lead time for proactive migration before forced eviction; and (3) a speculative decoding continuity engine that offloads draft model token generation to on-demand fallback nodes during migration windows, bounding the user-visible interruption to a sub-second buffering pause. MorphCloud-LLM is deployed and evaluated on AWS and GCP using LLaMA-70B and Mixtral-8x7B across 521 trace-injected preemption events, achieving up to 76% cost reduction under active-serving accounting (69.8% for LLaMA-70B; 67% including warm standby fallback capacity) with only 2.1% p99 latency overhead and zero dropped requests. Extensive ablation studies confirm the contribution of each component to overall system resilience. Note that preemption events are reproduced via a trace-driven simulation framework built on empirical AWS and GCP spot interruption traces rather than fully uncontrolled live production preemptions. Production generalizability under uncontrolled preemption—including simultaneous multi-node failures, network congestion, storage contention, and replacement-instance scarcity remains subject to future validation in sustained live deployments.