MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inference
This work proposes MaskCoFT, a masked co-adaptive fine-tuning method that trains routers and experts together with the cross-entropy loss alone, and reduces expert fetches per token in real offloading system serving.