MassAlloc Attention: Let Attention Allocate Its Own Compute
FullAttn often assigns negligible normalized mass to much of the causal score space, yet dense kernels execute the complete post-score path after forming each QK tile. We introduce MALA, a fused attention primitive that preserves score access to every legal causal interaction and uses normalized contribution to allocat...
Jing-Ze Shi, Zhang-Yang Peng, Xian-Duo Li et al.
· 0 citations