Xin-Song Ma

A Fine-Grained Analysis of the LoRA Fine-Tuning Landscape with Implications for Data Selection

Low-Rank Adaptation (LoRA) has become a standard approach for parameter-efficient fine-tuning, yet a fundamental practical question remains unresolved: how should the adapter rank be chosen? An overly small rank may lead to a poorly conditioned optimization landscape, whereas an unnecessarily large rank sacrifices the...