Unlocking More Granular Control of Memory-Efficient LLM Finetuning
This work systematically investigates the impact of the projection unit on LoRP methods, and extends existing LoRP approaches by introducing an additional degree of freedom, projection granularity, beyond the traditional rank hyperparameter, which enables a framework capable of performing fine-grained projections, which is named VLoRP.
Yezhen Wang, Zhouhao Yang, Fanyi Pu et al.
· 0 citations