Ryan A. Rossi

Octopus: Gated Selective Attention for Memory-Bounded Long-Context Inference in Large Language Models

O CTOPUS is proposed, a framework that confers fixed-memory inference onto pretrained Transform-ers without the information loss of linearization and outperforms state-of-the-art linearized baselines on the GSM8K benchmark, demonstrating that learned sparse retention serves as an effective regular-izer for long-horizon...