Ke-Qin Li

S-GPT: Shape-aware graph partitioning and tuning for sustainable inference of large-scale deep neural networks

The rapid growth of large-scale deep neural networks has pushed inference workloads toward increasingly diverse shapes. In practical inference, these models are often invoked with tensors of varying shapes rather than a single fixed shape. Such shape variability reshapes operator loops and intermediate tensors, leading...