UltraQuant: 4-bit KV Caching for Context-Heavy Agents
This work frames 4-bit KV caching around multi-round agent workloads where task quality, cache residency, and serving throughput must be measured jointly, and presents serving optimizations on AMD GPUs, including optimized decode-attention kernels and UltraQuant.