Analyzing the Impact of Architectural Design Decisions on Performance Across Generations of NVIDIA GPUs

Abstract

The high-performance computing industry is moving beyond an era in which each generation of GPU provides uniform performance gains across all applications. The growing importance of AI is driving GPU architecture towards greater specialization, with more silicon devoted to Tensor Cores and reduced-precision arithmetic. As a result, the benefits of new chip generations are increasingly dependent on the characteristics and computational demands of the workload being run. This project analyzes how the architectural changes introduced across NVIDIA’s Volta, Ampere, and Hopper GPU generations translate into real-world performance and energy efficiency gains. Experiments were conducted using one GPU per run to isolate device-level performance and avoid the effects of inter-GPU communication. The application suite included scientific computing, AI training, AI inference, and targeted microbenchmarks to stress different parts of the GPU such as matrix computation, memory bandwidth, and data movement. For each workload, we measured performance, execution time, energy consumption, and GPU hardware counters to compare application scaling with the architectural advancements introduced across each generation. The results show that performance improvements from Volta to Hopper range from 2.95x to 8.32x, demonstrating that generational gains depend strongly on workload characteristics. Matrix-heavy applications, including many modern AI workloads, achieved the largest improvements in both performance and energy efficiency by effectively using increasingly capable Tensor Core hardware. In contrast, memory-bound and vector-based workloads experienced more modest gains because memory bandwidth and general-purpose compute resources have scaled more slowly. Overall, the results show that specialization can deliver substantial performance and efficiency gains, but only when workloads can effectively use the hardware being added. Future work will extend the analysis to additional GPU generations and applications, while exploring frequency capping and more detailed die-area analysis. Supervisor: Ian Karlin