Preprint
Sep 2026
Trillion-Parameter MoE in a Box: Decoupling Memory Provisioning with High-Bandwidth Flash
An MoE appliance for trillion-parameter models at low concurrency must host terabytes of weights on one node and serve prefill and decode with fixed resources. Combining operator analysis of two trillion-parameter MoE models, a measured expert routing trace, and agentic serving traces over multiple turns, we explore a...
Peng-Fei Xia, Tuo Hao, Sheng-Wei Li et al.
· 0 citations