#artificial intelligence
Sep 2026
DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory
DPS is a dual-precision LLM serving system that turns weight memory into an elastic resource: under normal load, DPS serves the full-accuracy model; under KV pressure, it switches to a nested, lower-precision variant and repurposes unused weight memory for KV cache blocks.
Xuan Truong Nguyen, Tien-Son Pham, Tuan-Duc Chu et al.
· arXiv.org · 0 citations