Jul 2026
MoE2-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation
This paper makes the first attempt to fine-tune MoE models with MoE-style low-rank adaptation via a dual-channel Routing-Conditioned Projection module, which reuses base router activations to inform LoRA routing and introduces a single global LoRA expert pool shared across all layers.
Qingyu Yang, Haonan He, Minglei Li et al.
· arXiv.org · 0 citations