Rank-Preserving KV Cache Compression for Large Vision-Language Models
Advancement of Large Vision-Language Models (LVLMs) has enabled comprehension of long and high-resolution multi-modal contexts, but this capability introduces a significant Key-Value (KV) cache memory bottleneck during autoregressive generation. Existing KV cache compression methods typically rely on greedy top-k evict...