DeepShare is a scheduler that uses a continuous tenant-assurance signal to coordinate these decisions at runtime to achieve a more advantageous utilization-QoS trade-off than optimizing quotas, scheduling, and resource sharing independently.
Jing-Hao Wang, Yi-Hang Zhou, Xiao Zhou et al.· 0 citations
A prediction-guided runtime that uses workflow forecasts to construct and optimize a physical execution graph and reduces end-to-end makespan and overall p95 completion latency under burst arrivals by up to 36.8% and 25.8% over state-of-the-art workflow schedulers.
Jing-Hao Wang, Yi-Feng Zhang, Xiao Zhou et al.· 0 citations
This work presents SpecBox, a runtime built around speculative sandbox preallocation tailored for dynamic LLM agent execution pipelines, and implements keyword matching and streaming semantic embedding to enable intent-driven sandbox prewarming, which identifies pending tool execution demands mid-LLM token generation a...
Yi-Hui Zhang, Tian-Yu Wo, Jing-Hao Wang et al.· arXiv.org· 1 citation
Deep neural networks (DNNs) with billions of parameters power many important applications, but their training is fundamentally constrained by the limited on-chip memory of GPUs. This memory wall forces training to rely on distributed execution or memory offloading, both of which introduce substantial inefficiencies. Ex...
Xiaoyang Sun, Jie Xu, Zheng Wang· IEEE Transactions on Paralle...· 0 citations
ElastiCo is presented, an elastic co-location framework that enables training and inference workloads to safely share GPUs through three integrated mechanisms that decomposes the resulting multi-resource allocation problem into per-job configuration selection subproblems via dynamic per-resource shadow prices.
Jing-Hao Wang, Yi-Hang Zhou, Xiaoyang Sun et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.