Skip to content
Book Open access

Cadence: Taming Coupled Contention in Disaggregated Agent Clouds

Nov 2026 · Proceedings of the 2026 ACM Symposium on Cloud Computing Volume 1 · 0 citations · 31 references

Abstract

Cadence targets multi-tenant LLM serving over operator-declared phases on disaggregated GPU pools and remote KV/cache tiers. We assume a small phase taxonomy—retrieval, tool fan-out, structured parsing, and decode—and adapt resource parameters within each phase online, rather than discovering arbitrary new behaviors at runtime. Despite strong single-tenant optimizations, clusters miss tail-latency SLOs once multi-phase requests collide with co-tenant bursts across GPU, cache, and network. The core challenge is coupled contention: a scheduler that only balances GPU utilization can still create cache hot spots and cross-rack fan-out, while transport-only fixes cannot prevent cache thrash or admission overload. Cadence jointly allocates GPU compute, KV/cache residency, and network bandwidth for phase-structured requests under per-tenant SLO and fairness constraints. It combines phase-aware resource contracts, an online co-scheduler for placement/cache/bandwidth bundles, and guardrails that combine admission control with cache and network enforcement. Across a 32-GPU, 400 Gbps leaf–spine testbed running four templates with eight tenants, at 90% offered load Cadence reduces P99 latency by 1.4–1.8 × vs GPU-Only and cuts SLO violations by 6.0–7.6 × vs GPU-Only (5.1–7.1 × vs Cache-Only/Net-Only); at the GPU-Only saturation point (100% normalized offered load), it improves cluster goodput by up to 54%. Cadence remains stable under 50% tenant churn and 40% retrieval skew, maintains Jain’s fairness index ≥ 0.84 at full load, and incurs 2.25% total CPU overhead in the control plane. The supplemental artifact includes the code, configurations, and scripts needed to reproduce the figures and tables.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.