HyNA: Taming Tail Latency in MoE Training with Hybrid Switch Silicon

Abstract

The transition to trillion-parameter models, particularly Mixture-of-Experts (MoE), shifts the bottleneck of distributed training from computation to communication. However, existing Parameter Server (PS) architectures succumb to incast congestion, while state-of-the-art In-Network Aggregation (INA) solutions like ATP fail to handle the sparse, bursty traffic of MoE workloads. These solutions suffer from severe tail latency amplification due to their reliance on slow, host-based fallbacks for collisions and overflows. To dismantle this communication wall, we propose HyNA, a fully serverless aggregation system that eliminates dedicated parameter-server nodes by leveraging a novel hardware-software co-designed switch architecture. HyNA couples wire-speed Reconfigurable Match Tables (RMT) with embedded RISC-V cores. By adhering to a strict on-chip closure principle, the system processes all traffic anomalies—including hash collisions and floating-point variances—entirely within the switch ASIC, converting unpredictable network RTT into deterministic on-chip latency. We validate our design through a 100 Gbps FPGA prototype and a 7nm ASIC synthesis analysis. Results demonstrate that HyNA incurs less than 3% silicon area overhead while improving aggregation throughput by 7.35X over BytePS and 1.4X over ATP. Crucially, in the MoE gradient synchronization phase, the system eliminates the fallback penalty and reduces synchronization time by up to 1.6X compared to dynamic INA baselines, without compromising bit-level model accuracy.