Graphics Processing Units (GPUs) have been serving as critical computation resources for large-scale parallel computations. With increasing chip complexity, power efficiency has become an important design objective for modern GPUs. GPU power optimization relies on fast power evaluation, requiring architecture-level GPU power model. However, because of the time-consuming power label collection, only simple microbenchmarks are adopted for training. The limitation of microbenchmarks as training data incurs low accuracy for existing architecture-level GPU power models like AccelWattch. To address the limitation of microbenchmarks as training data, we propose G-Power, an architecture-level GPU power modeling framework that utilizes additional known GPU chips to provide additional knowledge. G-Power utilizes the aggregated knowledge foundation from additional known GPU chips and then performs fine-tuning on our target GPU. To provide foundations with additional known GPU chips and capture the similarity to utilize these foundations for fine-tuning, G-Power adopts a three-phase algorithm consisting of 1) pre-training with additional known chips, 2) attention-inspired aggregation, and 3) fine-tuning on our target GPU. We evaluate G-Power on four modern NVIDIA GPUs, demonstrating high accuracy. G-Power can achieve a low MAPE of 14% and a high correlation coefficient R of 0.88 on average, which are 22% lower MAPE and 0.36 higher R than AccelWattch.
Qijun Zhang, Yao Lu, Shang Liu et al.· 0 citations
With the growing demand of artificial intelligence (AI) applications, large language models (LLMs) have become important workloads in many domains. The question of how to efficiently generate optimal AI chip accelerator designs remains unresolved and challenging. Currently, there is a lack of end-to-end design methodologies for efficient design space exploration (DSE). We propose FSGen, an agile framework for attention-based LLM accelerator generation with an early-stage PPA estimator. FSGen supports fused operator dataflows and sparsity with a diverse design space and finds designs with 1.4x better power efficiency or 10x speedup with similar PPA metrics compared to prior work. Pareto-optimal designs have much better performance over a wide range of LLM benchmarks and have 58x better figures of merit (FoM). Design exploration is also faster due to our PPA estimators, which have better accuracy than prior art and reduce DSE runtime drastically.
With the growing demand for artificial intelligence (AI) applications, high-performance and energy-efficient AI chips are needed to support the computation. However, architectural-level AI chip design, PPA evaluation, and power estimation remain challenging due to the exponential set of possible designs and the difficulty of accurately modeling the impact of diverse dataflows and workflows on the underlying hardware at the architectural-design stage. We propose a novel open-source framework named OSCAR, which, given a set of hardware and workload specifications, provides architecture-level power estimation and can also automatically generate Chisel and synthesizable RTL of the custom AI chip. Our contributions include (1) a flexible and hierarchical AI chip design space, software and hardware stack, and an RTL generator supporting dense, Transformer, Winograd, systolic, and reconfigurable architectures in one unified framework, (2) hierarchy-based data-sensitive power model using architectural-level toggling features, achieving 3.8% error and correlation coefficient R > 0.99 to post-synthesis power, outperforming state-of-the-art power estimation methods, (3) validation of our power model by performing design space exploration, finding designs with better Pareto-optimality, 2.5 × lower power or 2 × better runtime metrics, compared with using prior art power models, (4) a tape-out of an AI chip based on DSE results, with OSCAR modeling its power with over 90% accuracy.
J. Mok, Qi-Jun Zhang, Di Pang et al.· ACM Transactions on Design A...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
