W.V.S.O. van den Dool

AVQ-Attention: Adaptive Vector-Quantized Attention

This work proposes Adaptive Vector-Quantized (AVQ) Attention, which adaptively allocates codebook capacity based on attention importance and develops an implementation using custom Triton kernels that enables the full adaptive refinement process, including importance scoring, child codeword insertion, and parent contribution replacement, to be carried out within the tiled computation paradigm of Flash Attention with minimal overhead.