Jul 2026· International Conference on Supercomputing· pp. 314-327· 0 citations· 54 references
Computer Science
TL;DR
This work introduces Calculon-MoE, a tool that jointly explores FLOPS, HBM bandwidth and capacity, multiple network topologies, and popular parallelism/optimization strategies used in LLMs, and quantifies the benefits of overlapping compute and communication, hardware-accelerated collectives, widening the scale-up domain, and higher memory bandwidth and capacity.
Abstract
The massive compute, memory, and networking needs for LLM training necessitate a fundamental rethinking of datacenter architectures to ensure scalability, efficiency, and cost-effectiveness. In particular, the design of the network fabric for AI datacenters for emerging LLMs (such as MoEs) remains a crucial and challenging open question, spanning technology choices (that determine the size of the high-bandwidth domain), topology, and software optimizations (collective algorithms and overlap strategies). This necessitates an agile framework to traverse the co-design space. This work introduces Calculon-MoE, a tool that jointly explores FLOPS, HBM bandwidth and capacity, multiple network topologies (Two-tiered vs. FullFlat optical), the size of scale-up domain, and popular parallelism/optimization strategies used in LLMs. Our validation studies demonstrate that our LLM/MoE runtime predictions are within 10% of real-world measurements. Using Calculon-MoE, we conduct a suite of case studies to develop an actionable roadmap for data centers. For example, the results point to the promise of Fullflat network architectures, which provide uniform high-bandwidth, low-latency connectivity between all nodes and demonstrate their positive impacts on performance and scalability. We also quantify the benefits of overlapping compute and communication, hardware-accelerated collectives, widening the scale-up domain, and higher memory bandwidth and capacity. Our study spans both sparse (mixture of experts) and dense transformer-based LLMs, revealing how system design and optimization choices affect system efficiency and overall throughput in both cases.
Google's Global Network (GGN), a major architectural redesign of the WAN that evolves B2 and B4 into a single, modular, and highly available software-defined network, is presented.
Mohammad Al-Fares, R. Alimi, Arda Balkanay et al.· Conference on Applications,...· 0 citations
This work presents Meta's network architecture and software stack designed to support one of the world's largest RoCE fabrics, currently connecting over 100,000 GPUs across multiple datacenter buildings, and introduces a scalable initialization strategy that reduces startup times by 11× via eager process group creation and O(N) topology discovery.
Hongyi Zeng, Min Si, Pavan Balaji et al.· Conference on Applications,...· 0 citations
The deployment of Mixture-of-Experts (MoE) models on production high-bandwidth superpods, such as NVIDIA's NVL72/576 and Huawei's CloudMatrix384, introduces critical challenges beyond raw interconnect bandwidth. While these systems provide unified global address spaces and high-bandwidth fabrics, their full potential for sparse MoE communication is hindered by three fundamental bottlenecks: (1) Strict execution serialization imposed by coarse-grained Bulk Synchronous Parallel (BSP) orchestration of interdependent communication phases; (2) Prohibitive synchronization overhead that fails to scale alongside high interconnect bandwidth; and (3) Severe load imbalance resulting from distance-agnostic scheduling of irregular token traffic. To eliminate these bottlenecks, we introduce UBEP (Unified-Bus Expert Parallelism), a production-ready communication library that rethinks MoE's All-to-All primitives for modern superpod architectures. Through large-scale experiments, UBEP reduces All-to-All latency by up to 52.4% and MoE inference Time Per Output Token (TPOT) by up to 11.1%.
Yipeng Liu, Chang Liu, Sitan Shen et al.· Proceedings of the ACM SIGCO...· 0 citations
Networking research advances by turning hypotheses into empirical evidence, so accelerating it means reducing the lag between ideation (synthesizing a hypothesis) and generating the data that tests it. Consider a concrete case: does a bulk BBR download fairly share its bottleneck with competing real-time Google Meet traffic? Validating this requires configuring a realistic bottleneck link, concurrently generating BBR's bulk transfer and Meet's real-time traffic, and collecting relevant service-quality metrics. Today this overhead is high, often forcing researchers to start from scratch for every new idea. This ideation-to-data-generation gap will only worsen in the agentic AI era, where AI-assisted ideation accelerates exponentially, yet its outputs cannot be validated without a data-generation backend. This paper explores how to bridge this gap. We envision a composable, domain-specific backend, Pramana, shaped as a thin waist, with diverse research intents at the top and disparate execution substrates at the bottom. Pramana realizes this waist through a single contract, the intent specification, which disaggregates an experiment into three independent axes: the intent (what data to generate), the substrate (where to generate it), and the mechanism (how to produce it), so one specification runs on any substrate. We demonstrate Pramana's utility by building a first-of-its-kind corpus of 255 data-generation intents mined from 66 published papers, and show the intent specification satisfies all of them, where no existing tool satisfies more than 13%. Our current proof-of-concept implementation already satisfies 34% of these intents, more than twice the best existing tool, and we lay out a roadmap for closing this abstraction-implementation gap through a broader community effort to build the envisioned data-generation backend and accelerate empirical networking research.
Jaber Daneshamooz, Eugene Vuong, A. Ramanathan et al.· 0 citations
These findings demonstrate that deployment decisions based on structural failure domains can substantially limit the propagation of failures beyond the intended canary population, and indicate that topology-aware rollout control and per-datacenter monitoring can substantially reduce failure propagation and containment time relative to topology-blind global monitoring.
Sai Joshitha Kathari· American Journal of Technolo...· 0 citations
DistDPU is presented, a disaggregated DPU architecture that redefines the scaling abstraction for high-bandwidth cloud networking and co-designs the EM-OM functions and the inter-module fabric to minimize virtualization overhead while enforcing security and manageability invariants equivalent to those of a monolithic DPU.
Hao Mei, Lizhou Gao, Yuanyi Zhu et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
