Han-Ming Zhang

COPE: Composable Performance Emulation for Mixture-of-Experts Offloading Policies

As LLMs scale, mixture-of-experts (MoE) architectures increase model capacity while keeping per-token computation manageable through sparse expert activation. This sparsity also enables memory-efficient deployment by retaining most expert weights in CPU memory and transferring or executing only the routed experts on de...