FoldAttention: Declared-Reference Softmax for Fast Decode and Deterministic Backward
FoldAttention is presented, an additive formulation of softmax attention that fixes a finite reference $Z_i$ before scanning the KV cache and removes the determinism tax.
Sriman Achanta
· 0 citations