H3-Attn: Attention-Efficient 3D DRAM PNM Processor with Hybrid Head Parallelism and Hierarchical SFU
H3-Attn is proposed, an Attention-efficient 3D DRAM PNM processor for low-batch LLM inference that features a hybrid head parallelism for Attention processing, whereby various optimized Attention mechanisms with spatial tiled FlashAttention can be flexibly enabled with fully leveraged 3D DRAM PNM bandwidth.