HyNA: Taming Tail Latency in MoE Training with Hybrid Switch Silicon
HyNA is a fully serverless aggregation system that eliminates dedicated parameter-server nodes by leveraging a novel hardware-software co-designed switch architecture, and reduces synchronization time by up to 1.6X compared to dynamic INA baselines, without compromising bit-level model accuracy.