Aug 2026· Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication· 0 citations· 41 references
Computer Science
TL;DR
A Aegis scheduler is presented, a scheduler that adapts placement using live fabric telemetry under operator-defined contracts that reduces service p99 RPC latency, cuts SLO violations, and lowers ECN mark rate while improving utilization under production-derived workloads.
Abstract
Accelerator clusters generate bursty network traffic: distributed training exchanges gradients via all-reduce, inference pipelines fan out requests, and storage competes with latency-sensitive RPCs. Poor placement amplifies congestion: a mis-scheduled training job can raise ECN mark rates by 3× and inflate co-located service tail latencies by 40%. Yet schedulers ignore real-time fabric state. We present Aegis, a scheduler that adapts placement using live fabric telemetry under operator-defined contracts. Aegis ingests per-ToR queue depths and ECN counters at 1-second granularity, incorporates a congestion-pressure term into placement scoring, and evolves online via a conservative bandit with exposure control. Contracts on tail latency, fairness, and congestion compile into safety envelopes that gate rollouts and trigger automatic rollback. We evaluate Aegis on an 80-node cluster (160 A100-80GB GPUs, 160 TPU v4 chips, 2-tier Clos fabric). Under production-derived workloads, Aegis reduces service p99 RPC latency by 27.3% (152.4 ms vs. 209.6 ms), cuts SLO violations by 3.6×, and lowers ECN mark rate by 51% while improving utilization from 68.4% to 79.1%. These gains hold under workload shifts and telemetry delays up to 60 seconds. The design is scoped to shared clusters where placement can materially reshape fabric contention.
ElastiCo is presented, an elastic co-location framework that enables training and inference workloads to safely share GPUs through three integrated mechanisms that decomposes the resulting multi-resource allocation problem into per-job configuration selection subproblems via dynamic per-resource shadow prices.
Jing-Hao Wang, Yi-Hang Zhou, Xiaoyang Sun et al.· 0 citations
SemaCredit is presented, a receiver controller that admits each remote-memory operation against a vector of target-resource demands and returns each component when its corresponding HBM, Atomic, or response stage completes, reducing small-operation P99 latency by 52.4% under Atomic contention and 10.2% under response incast.
Fan Yang, Jiaqi Liu, Tao Jiang et al.· 0 citations
LazyTrain is proposed, an optimization layer over a layer-streaming executor that formulates checkpoint selection, activation placement, recomputation, and CPU-GPU-NVMe communication overlap as a mixed-integer scheduling problem, then executes the solved policy during training.
Xiao-Jun Wu, Cehao Yang, Honghao Liu et al.· 0 citations
TurboBus is presented, which pools PCIe bandwidth across co-located jobs via emerging scale-up fabrics and reduces first-token latency by up to 40% for on-demand model loading, achieves up to 1.6x throughput for KV-cache-offloaded inference, and accelerates training by up to 7%, while imposing less than 1% overhead on co-located workloads.
Xinyu Yang, Kaiqiang Xu, Kai Chen· Conference on Applications,...· 0 citations
The method lets an HPC site compare deployment strategies using both user-visible performance and scheduler impact, then select the fastest strategy within its own RPC-demand limit.
Nil Tianchen Mu, William Dizon, Glen Otero et al.· 0 citations
AI and HPC infrastructure increasingly serves workload portfolios that combine dense tensor computation, sparse kernels, large memory footprints, and communication-intensive collectives. Supporting these portfolios requires coordinated choices across accelerators, memory tiers, scale-up fabrics, and cluster networks. The resulting Cross-layer Heterogeneous System (XHS) design space is difficult to explore: hardware choices change legal task mappings, while rack power, switch radix, cabling, and cost constraints invalidate many candidates. We present CHASE, an application-driven framework that searches physically feasible XHS architectures through the workloads they must execute. CHASE represents candidates as hierarchical typed graphs and rejects designs that violate deployment constraints. It avoids intractable joint hardware-mapping search with a decoupled two-level loop: an inner mapper translates hardware-independent workload DAGs into topology-aware event traces, a calibrated event-driven simulator evaluates each mapping, and an outer telemetry-guided optimizer evolves the hardware graph. We evaluate CHASE on sparse-computing and LLM workloads. Its mapper remains within 6.06% of exhaustive optima while reducing mapping time by 60.5% on average relative to PEFT. Compute-model errors average 4.4-7.5%, and communication validation reproduces key trends across physical platforms. The outer search reaches near-global optima within 64 iterations. End-to-end case studies show that sparse workloads favor criticality-aware heterogeneous pods, whereas LLM inference favors scale-up islands; the resulting designs deliver 6.20$\times$ and 2.12$\times$ geomean speedups, respectively, while reducing cost and power relative to the baselines.
Yuchen Fan, Minghong Sun, Jikui Ma et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
