Book
Open access
Aug 2026
Connex: Endpoint Mobility Primitives for Dynamic LLM Serving
Evaluation on a 5-node GPU cluster under synthetic and production-derived churn shows that Connex reduces P99 tail spikes by up to 85% compared to NCCL-based baselines, achieves sub-second cutover, and maintains 100% goodput at moderate loads where baselines collapse to 0–28%, while incurring less than 5% steady-state overhead.
Yanying Lin, Vincent Liu, Tao Luo et al.
· Conference on Applications,... · 0 citations