Efficient GPU Multitasking with Morphable Kernels
TL;DR
MorphX is a GPU multitasking system that embraces the runtime adjustability of kernels as the first principle, and transparently transforms traditional kernels into morphable ones at compile time, and leverages an in-place profiling mechanism to orchestrate fine-grained scheduling based on real-time resource contention.
Abstract
GPU multitasking offers a promising approach to improving hardware utilization by co-locating concurrent workloads on the same device. However, achieving high resource utilization with minimized interference requires fine-grained, adaptive scheduling. Existing scheduling solutions are fundamentally constrained by a rigid assumption: once a kernel is launched, its resource footprint remains fixed throughout its execution. Consequently, they either rely on static resource pre-partitioning, which lacks the flexibility to adapt to rapid workload changes, or adopt kernel-slicing techniques, which achieve fine-grained control at the cost of prohibitive kernel launch overheads. In this paper, we break this constraint by introducing morphable kernels, a new software abstraction that allows a kernel to dynamically yield or reclaim compute resources on the fly without additional launches. Building on this abstraction, we design MorphX, a GPU multitasking system that embraces the runtime adjustability of kernels as the first principle. MorphX transparently transforms traditional kernels into morphable ones at compile time, and leverages an in-place profiling mechanism to orchestrate fine-grained scheduling based on real-time resource contention. Evaluation under various scenarios shows that MorphX can improve background task throughput by 2.24x under the same foreground task latency, compared to existing systems.