PRISM-LoRA: Principal-Direction-Guided Single-LoRA Merging Framework for Continual Learning of Large Language Models
Continual learning of large language models requires incorporating new knowledge while limiting catastrophic forgetting; however, many low-rank adaptation (LoRA)-based methods retain task-specific modules or separate learning spaces that grow with the task sequence. We propose PRISM-LoRA, a replay-free framework that r...
Taehyeong Kwon, O. Jeong
· Mathematics · 0 citations