#artificial intelligence
Jun 2026
UltraQuant: 4-bit KV Caching for Context-Heavy Agents
This work frames 4-bit KV caching around multi-round agent workloads where task quality, cache residency, and serving throughput must be measured jointly, and presents serving optimizations on AMD GPUs, including optimized decode-attention kernels and UltraQuant.
Inesh Chakrabarti, David Limpus, Aditi Rana et al.
· arXiv.org · 0 citations