Skip to content

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Book Open access Nov 2026

MosaicKV: SLO-Aware KV-Cache Virtualization for Multi-Tenant LLM Serving in the Cloud

Large language model (LLM) serving has become a first-class cloud workload. The dominant cost driver is no longer compute alone. The key-value (KV) cache consumes tens of gigabytes per replica. It forces providers to either over-provision GPU memory or reject long-context requests. Existing high-throughput engines larg...

Rui Li, Shuang Cao · 0 citations
Book Open access Nov 2026

Cadence: Taming Coupled Contention in Disaggregated Agent Clouds

Cadence targets multi-tenant LLM serving over operator-declared phases on disaggregated GPU pools and remote KV/cache tiers. We assume a small phase taxonomy—retrieval, tool fan-out, structured parsing, and decode—and adapt resource parameters within each phase online, rather than discovering arbitrary new behaviors at...

Rui Li, Shuang Cao · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.