Aug 2026· Asia-Pacific Workshop on Networking· 0 citations· 38 references
Computer Science
TL;DR
SprayCast is proposed, a congestion-adaptive native RDMA multicast scheme for dynamic sparse token Dispatch that uses in-band network telemetry feedback to steer replication away from congested multicast branches and range-based negative acknowledgments for localized loss recovery, saving bandwidth and reducing tail latency in dynamic all-to-all communication.
Abstract
Mixture-of-Experts (MoE) models outperform traditional dense models through sparse expert activation, where each token is dynamically routed to a small subset of experts. Across many tokens, these sparse Dispatch operations induce all-to-all traffic, making communication a major bottleneck for both training and inference: unicast replication wastes bandwidth, while table-driven multicast struggles with receiver-set churn and incast. In this paper, we propose SprayCast, a congestion-adaptive native RDMA multicast scheme for dynamic sparse token Dispatch. To avoid maintaining multicast forwarding tables in switches, SprayCast encodes each packet’s destination node set in its packet header using hierarchical bitmaps, enabling table-free in-network replication. It uses in-band network telemetry (INT) feedback to steer replication away from congested multicast branches and range-based negative acknowledgments (NACKs) for localized loss recovery, saving bandwidth and reducing tail latency in dynamic all-to-all communication. In htsim simulations on a 128-server fat-tree, SprayCast achieves better scalability as top-K dispatch fanout increases and reduces P99 dispatch tail latency by up to 6 × at K = 8 compared with representative baselines.
LLM deployment is migrating from data centers to edge devices, where Mixture-of-Experts (MoE) models offer a promising path: sparse expert activation allows the model to be spread across multiple low-cost edge nodes. Distributed MoE inference repeatedly dispatches embeddings from one main node to many workers - a one-to-many pattern poorly served by the sequential unicasts of mainstream stacks (NCCL, TCP), yet naturally matched by UDP broadcast. We propose a UDP broadcast method for collaborative edge MoE inference, augmented with timeout-driven retransmission exploiting near deterministic latency in distributed MoE for reliability and unordered result gathering for robustness to expert mispredictions, yielding a consistent 1.4x speedup over NCCL and TCP on a wired 8-node cluster. In wireless settings, however, we uncover a deeper, long-forgotten bottleneck: IEEE 802.11 caps broadcast rates at 54 Mbps regardless of physical-layer capacity - a legacy policy built for sparse control traffic, not edge AI. NS-3 simulations at distances 1m, 2m and 5m show that the optimal rates are much higher (64x, 43x, and 32x, respectively) than the 54 Mbps cap applied in standard. Thus, we argue that broadcast is no longer a control-plane relic: it is time for Wi-Fi standards to treat it as a high-throughput data-plane citizen.
Liujianfu Wang, Yuyang Du, Shiqi Xu et al.· 0 citations
Modern CDN vendors typically provide multi-server services for content deliveries, in which different requests from the same client are often scheduled to diverse CDN nodes. In this case, the requested data (e.g., short video chunks) always suffer from cold start during its transmission, especially for the first flow of established connections, due to the unknown client-side network quality. Existing schemes focus on single-server startup parameter optimization, whose referred metrics appear to be extremely limited. To address this issue, we propose hStart, a novel collective service paradigm that enables hot-start congestion control by efficient collaboration between CDN nodes. hStart synchronizes probed network quality information (e.g., bandwidth, latency) across CDN nodes to initialize congestion control from historical data, avoiding probing convergence. It uses hierarchical state synchronization to limit overhead, and adaptive prediction with a time-decay trust model to handle dynamics. We implement and deploy the hStart system for the QUIC protocol. Extensive evaluations in both controlled testbeds and large-scale production CDN environments demonstrate that hStart effectively eliminates cold start penalties, boosting 46.47% throughput gain and 21.43% rebuffering reduction.
Tong Li, Jiuxiang Zhu, Bo Wu et al.· Asia-Pacific Workshop on Net...· 0 citations
Probabilistic state-proportional (PSP) dispatching, a packet-level load balancing algorithm using a Band-based discrete state representation, provides an effective balance among performance, stability, and overhead for artificial intelligence data centers.
Jiaqi Liu, Chunyang Zhang, Heng Pan et al.· 2026 IEEE 51st Conference on...· 0 citations
ProLet is a load balancing scheme that enables proactive probing and reroutes elephant flows at flowlet granularity in lossless RDMA networks and reduces average and tail flow completion time slowdowns by 69% and 79%, respectively, compared to state-of-the-art load balancing schemes.
Hong Wang, Jin-Hao Luo, J. Tan et al.· Asia-Pacific Workshop on Net...· 0 citations
Volumetric streaming remains difficult to scale because receivers with overlapping fields of view are often served independently, causing repeated transmission of shared content. We present MD2G-Cast, a relay-coordinated multicast framework over Media over QUIC with an application-aware control layer for scalable multi-user volumetric delivery. MD2G-Cast jointly uses viewing overlap, receiver capability, and bandwidth conditions to form reusable multicast groups, share common Base content, and selectively admit Enhanced delivery. We formulate grouping and Enhanced admission as a sequential control problem, realize it with Proximal Policy Optimization (PPO), and train a compact relay model with teacher guidance for Enhanced admission. We implement MD2G-Cast with real MoQ processes and evaluate it with real access and 6DoF viewing traces for up to 100 users. At 20 and 100 users, MD2G-Cast keeps the receiver-side $P_{99}$ delivery interval below 40 ms across all seven access profiles, while Rolling reaches the 500 ms reporting cap in most cases. Across the evaluated user scales, MD2G-Cast achieves the highest or tied-highest mean system utility under homogeneous access and the highest mean utility under heterogeneous access, while reducing aggregate link load by about 27% relative to Clustering at 100 users. A matched relay-control ablation separates the control structure from its optimizer, showing that random feasible actions reduce utility while deterministic control remains competitive with PPO. Together, the results support relay coordination and selective Enhanced admission, rather than a particular policy optimizer, as the central design contribution.
Ruonan Chai, Yisu Wang, Zili Meng et al.· 0 citations
This work shows that efficient offline-optimized routing enables efficient MoE training and inference on direct-connect topologies without the need for MoE traffic matrix or dynamic topology reconfiguration.
Ori Cohen, Jakob Krebs, Daniel Amir et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
