Preprint
Sep 2026
Adaptive Context Parallelism for Production LLM Serving
Vertumnus is presented, an adaptive CP serving system designed for heterogeneous and evolving workloads that reduces mean TTFT and improves token-weighted SLO attainment and introduces a global prefix-cache management policy that coordinates cache placement and replication among workers with the same or different CP de...
Jiarui Guo, Rong-Le Wang, Pei-Jun Huang et al.
· 0 citations