HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding
HiKV is a novel algorithm-hardware co-design that exploits KV cache redundancy through hierarchical importance awareness and outperforms state-of-the-art importance-based methods by achieving an additional 1.87x reduction in external memory accesses.