Ghost is an OS-level GPU virtualization layer integrated directly into the open-source GPU driver, using a GPU container abstraction with cgroup -like APIs for compute and memory control and privileged hardware-level scheduling and preemption for dynamic compute resource management.
GPUs are expensive resources but often underutilized. This inefficiency is exacerbated by the increasing diversity of AI workloads, where small or specialized models often fail to saturate high-capacity devices, necessitating GPU sharing. Existing sharing approaches force a hard tradeoff. They either partition hardware...
Yi-Cheng Liu, Yi-Fan Qiao, Tian Xia et al.· Proceedings of the 32nd ACM...· 0 citations
GPUs are increasingly in demand for AI workloads, yet often remain substantially underutilized, motivating workload colocation. However, colocation introduces interference that can degrade latency-critical workloads. Existing approaches mitigate interference using either heuristic-based scheduling or interference predi...
F. Strati, Ethan Graham, Leo Stephan et al.· 0 citations
Modern GPUs are no longer physically symmetric. Die scaling leads to both manufacturing-driven floorsweeping and cache and memory partitioning. The former creates chip-specific compute topologies, while the latter causes non-uniform memory access. These asymmetries are substantial. Topology-oblivious compute unit alloc...
X. Fan, Jia-Hao Wang, Wei-Hao Cui et al.· 1 citation
Job failures are frequent in large-scale GPU clusters for LLM workloads, leading to significant resource wastage. The vast majority of these are shallow disruptions (e.g., software errors or updates), where only the worker process crashes while the underlying GPU and OS kernel remain intact. Existing recovery systems,...
Hao-Yi Ma, Shi-Wei Gao, You-Min Chen et al.· Proceedings of the ACM SIGOP...· 0 citations
This work presents EMA, a memory sharing system that allows GPUs within a server to borrow and reclaim memory from each other, forming an elastic pool of capacity, and ensures performance transparency for both borrowers and lenders.
Yi Xu, Tian Xia, Ion Stoica· arXiv.org· 0 citations
This paper presents a zero-overhead, multi-vendor GPU compilation framework built natively into the Rust compiler (rustc) and LLVM backends, and leverages Rust's rich type system, ownership system, and strict aliasing guarantees to efficiently manage and optimize data transfers through LLVM's Offload infrastructure.
Manuel S. Drehwald, Marcelo Domínguez, Kevin Sala et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.