Performance Efficiency Trade-offs in LoRA and QLoRA-Based Fine-Tuning of Large Language Models

Abstract

The role of pre-trained language models is fundamental for NLP; however, fine-tuning them for a particular task becomes increasingly computationally costly as the size of the model increases. Approaches like LoRA and QLoRA are more cost-effective because they train just a handful of parameters, leaving the whole pre-trained base model relatively unaltered. How much these approaches preserve the performance of the model on a downstream task and how much the LoRA rank affects the efficiency-performance trade-offs become the matter of interest when doing research within certain hardware constraints. The aim of this project is to compare LoRA and QLoRA approaches under the same experimental setup. The experimental methodology entails the adaptation of the pretrained meta-llama/Llama-2-7b-hf model for a binary sentiment classification task based on the dataset with 2,000 samples of the Stanford Sentiment Treebank 2 (SST-2) dataset. LoRA and QLoRA are tested with ranks 4 and 8; other major experimental settings are fixed to ensure identical conditions. The performance of the adapted models is measured via the number of trainable parameters, maximum memory usage, training time, and checkpoint size.