CoRe-MoE: Compact Reusable MoE for Continual Multimodal Instruction Tuning
CoRe-MoE is proposed, a Compact Reusable MoE framework for parameter-efficient continual multimodal instruction tuning that improves final average performance over the strongest competing baseline by up to 5.90 points, while using less than 1% of the trainable parameters required by sequential LoRA for later tasks.
Run-Ze Liu, Naibin Gu, Ming-Xu Ai et al.
· 0 citations