Skip to content
Open access

Exploring Latency Hiding Strategies for GPU GEMM Using Pipelined Multiword Arithmetic

Oct 2026 · International journal of parallel programming · Vol 54 · 0 citations · 14 references

Abstract

Matrix multiplication is at the heart of many applications in AI, scientific computing, and quantum simulations, for which GPUs have been widely adopted due to the immense compute power and hardware acceleration they provide for such computations. In this paper, we explore the performance profile of computing dense matrix–matrix multiplications (GEMM) on GPUs by decomposing the operation into several cascading lower-precision GEMMs and exploiting the then-available overlap between data transfers and mixed-precision compute and measure its performance when compared to standard monolithic or tiled GEMMs. The proposed implementation exploits the high performance of low-precision matrix multiply-accumulate (MMA) operations on modern GPUs and their multi-stream execution capabilities to overlap multiple low-precision GEMMs with data transfers.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.