How Finite-Rank Bottlenecks Shape the Low-Rank Adaptation Landscape
Low-rank adaptation (LoRA) has become the standard method for parameter-efficient fine-tuning of large pretrained models, yet theoretical explanations for why low-rank updates suffice remain incomplete. Most existing analyses rely on the Neural Tangent Kernel (NTK) regime, which linearizes the network around the pretra...
Long Nguyen-Chi, Quynh Nguyen, Thanh Nguyen-Cung et al.
· 0 citations