Structure-Derived Bottleneck-Aware Scheduling for Multitasking MCM-GPUs

Abstract

Multi-chip-module (MCM) GPUs improve scalability by integrating multiple GPU chiplets, but spatial multitasking introduces severe performance interference due to non-uniform memory access and asymmetric inter-chip bandwidth. Existing GPU multitasking schedulers rely on runtime profiling metrics to guide placement decisions. These reactive approaches observe the consequence of interference rather than its structural cause, vary across inputs and co-running workloads, and incur non-trivial overhead. This paper presents SA-Scheduler, a structure-derived bottleneck-aware scheduling framework for multitasking MCM-GPUs. Instead of profiling execution behavior, SA-Scheduler derives hardware bottleneck regimes directly from kernel structure prior to execution. Through offline architecture-aware modeling, we extract iteration-normalized features such as warp-level working-set footprint and global memory intensity (GMI). A fine-tuned lightweight code model (Qwen-2.5-Coder-1.5B) assists semantic structural extraction for CUDA kernels. These features are translated via an analytical bottleneck model into distinct interference regimes (C, M1, M2, and M3), characterizing how kernels interact with hierarchical and asymmetric memory resources under co-execution. Based on the inferred regimes, SA-Scheduler determines chip placement without hardware modification or runtime bottleneck profiling. For streaming multiprocessor (SM) allocation and optional power-gating decisions, SA-Scheduler further employs lightweight counter-based feedback to tune resource usage under bounded overhead. Across 105 two-program workloads, SA-Scheduler improves system throughput (STP) by 17.63% and 10.48% over intra-chip and inter-chip scheduling, respectively, while increasing fairness by up to 33.27%. An energy-aware Power Mode further reduces average power by up to 25.39% under bounded throughput degradation. These results demonstrate that structure-derived bottleneck inference provides a principled and scalable foundation for multitasking on future MCM-GPUs.