SketchSSM: Write to the Full State, Read from a Compact Sketch
SketchSSM is introduced, which preserves full-state updates while approximating reads in hybrid-attention models, and preserves recall on four RULER retrieval tasks.
Omin Kwon, JoongWon Shin, Minseo Kim et al.
· 0 citations