Fine-Tuning a KV Cache Concatenation-Aware Model or Recomputing KV Caches? Why Not Both?
A combined approach is proposed that fine-tunes the model while taking KV cache concatenation into account and selectively recomputes a subset of the KV caches and demonstrates improved accuracy for long-context inputs.
F. Tachibana, Daisuke Miyashita, Jun Deguchi
· 0 citations