A Dynamical Theory of LoRA in Continual Learning
Despite the widespread use of Low-Rank Adaptation (LoRA), little is known about its dynamics in continual learning and the mechanisms by which low-rank updates affect catastrophic forgetting. We provide an asymptotically exact dynamical characterization of LoRA in a solvable two-task teacher-student model. In the high-...
Théo Marchetta, F. Alessandroni, Alessandro Breccia et al.
· 0 citations