PQ-HSA: Reusing Product-Quantized Scores for Hybrid Sparse-Approximate Attention
PQ-HSA (hybrid sparse-approximate attention) attends the selected tokens with their original keys and values, and the unselected tokens, the background, enter the same softmax through those scores, summed per inverted list and multiplied by the list's mean value.