Namhoon Lee

Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts

Mixture-of-Experts (MoE) architectures allow frontier language models to scale to trillions of parameters, but their deployment is constrained by massive memory footprints and memory-bandwidth limitations. Although modern accelerators provide Sparse Tensor Cores (SpTCs) that reduce weight storage and increase throughpu...