Олександр Сташук

Efficient FlashAttention on Blackwell via Fixed-Shift Softmax and Persistent Scheduling

On Blackwell, normalization and operand movement can limit attention kernels whose matrix multiplications are already deeply pipelined. We implement an FA4-style pipeline in Triton TLX with fixed-shift dense softmax and a saved inverse denominator for backward. The fixed shift removes recurrent accumulator corrections;...