Large-scale AI clusters impose higher requirements on network scalability, cost, and communication efficiency. The traditional Clos topology suffers from superlinear cost growth when scaling to over 100k GPUs, while the more cost-effective Dragonfly+ introduces "down-up" detours, deadlock risks, and complex routing design. This paper presents Dragonfly-Ultra, a scalable, low-cost network architecture for high-performance AI clusters. Dragonfly-Ultra can scale to over 260k GPUs with only 82% cost and 81% power consumption of a 3-layer Clos architecture. Dragonfly-Ultra optimizes inter-group connectivity to eliminate intra-group detours entirely. Beyond the topological benefits, Dragonfly-Ultra incorporates three key mechanisms to further improve network performance and optimize collective communication, including lightweight dual-waterline adaptive routing for fast congestion mitigation, virtual-link-based deadlock avoidance with lower hardware overhead, and uniform affinity-aware rank placement for balanced inter-group traffic across all phases. Simulation results on a 4k-node cluster show that, compared to Clos, Dragonfly-Ultra achieves up to 18.8% and 39.2% lower completion time for AllReduce and AlltoAll, respectively. Compared to Dragonfly+, the reductions are up to 27.9% and 62.1%, outperforming current mainstream topologies.
Rui Zhuang, Hui Yuan, Junye Zhang et al.· Proceedings of the ACM SIGCO...· 0 citations
Prefix caching has become a key technique for LLM serving, and nowadays the reusable KVCache contents are often hosted on distributed servers. For long-context LLM inferences with high cache hit ratio, cross-server KVCache transmission has become an emerging performance bottleneck; such network-intensive LLM inferences are increasingly prevalent in the coming era of agentic AI. However, existing LLM inference engines are essentially compute-centric; we find that they are highly inefficient when serving such workloads due to compute-stage service blocking and ignorance of KVCache-transfer cost. To efficiently serve network-intensive LLM inferences, in this paper, we design Sanic, an optimized LLM engine that treats KVCache transmission as a first-class citizen. Viewing KVCache loading and computation as equally-significant stages, Sanic decouples their service control and allows each stage to progress autonomously in an asynchronous manner, thereby improving the overall resource utilization. Moreover, when scheduling competing LLM inferences, Sanic treats the KVCache loading delay as an independent factor in service cost modeling, which is more accurate and can yield better scheduling decisions. Our testbed experiments with diverse benchmarks show that, Sanic can substantially enhance the service efficiency of network-intensive LLM inferences, improving the SLO-attainment by up to 61.67%.
Weiye Wang, Chen Chen, Junxue Zhang et al.· Asia-Pacific Workshop on Net...· 0 citations