Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches
Fathom is presented, a key scan in which each query decides how many bits of each key channel to read, and on real coding-agent sessions Fathom reaches the step agreement of the most accurate 136-bit scan at 92 bits.
Vivek Kalyanarangan
· 0 citations