Book
Open access
Sep 2026
LLM KV-cache: To Restore or To Recompute, That Is the Question
The challenges of the restoration-recomputation trade-off are investigated and its impact on inference performance when left unaddressed, and an I/O-aware KV-cache management policy is presented that dynamically navigates this trade-off.
Amirhossein Najafizadeh, Vasily Tarasov, Alex Merenstein et al.
· Proceedings of the 18th ACM... · 0 citations