Efficiency Comparison of LoRA and QLoRA for Indonesian Sentiment Analysis Using IndoBERT

Abstract

Sentiment analysis is an important task in Natural Language Processing (NLP) because it enables researchers to understand public opinion. Transformer-based models such as BERT have demonstrated strong performance in sentiment analysis tasks. To further improve model performance, a full fine-tuning (FFT) process is typically required to adapt the model to a specific task. However, FFT requires updating all model parameters, resulting in high computational and memory costs. To address this limitation, parameter-efficient fine-tuning (PEFT) methods such as Low-Rank Adaptation (LoRA) and Quantized LoRA (QLoRA) have been proposed. This study evaluates the effectiveness of LoRA and QLoRA for Indonesian sentiment analysis. Model performance is assessed in terms of accuracy, convergence speed, and GPU memory usage. Experiments were conducted on three IndoBERT variants using a social media dataset related to the Free Nutritious Food program in Indonesia. Experimental results show that LoRA and QLoRA achieve strong performance, reaching an accuracy of up to 90% and a macro F1-score of 89.96%, while reducing GPU memory usage by 40%-65% compared to FFT, particularly on IndoBERT-P2. QLoRA provides the fastest convergence, although its performance tends to decline at higher ranks. The findings also reveal a trade-off between model accuracy, memory efficiency, and convergence speed across different rank configurations. Overall, both methods have proven to be effective and efficient approaches for fine-tuning Indonesian language models on resource-constrained devices.