BR-MoE: A Memory-Budget-Aware Co-Design Framework for Efficient MoE Inference on Resource-Constrained GPUs
BR-MoE, a memory-budget-aware co-design framework that treats per-expert bit-width and compensator rank as joint variables under a single memory constraint, establishes a superior memory–accuracy trade-off over state-of-the-art quantization methods, delivers substantial end-to-end inference speedups, and re-derives an...