PALRec: Large Language Model-Based Sequential Recommendation with Parameter-Preserving Augmentation

Abstract

Large Language Models (LLMs) have demonstrated remarkable general-purpose abilities across a wide range of domains, and these strengths have also been increasingly evidenced in recommender systems. However, existing methods that attempt to integrate collaborative signals into LLMs often fail to preserve their foundational knowledge. This loss is critical in text-rich recommendation, where robust semantic understanding is required to interpret user reviews and item profiles. We propose PALRec, a parameter-preserving augmentation framework that equips an LLM with recommendation capabilities while keeping its original parameters fixed. We first construct evidence-grounded user and item profiles from reviews and use them as concise pseudo-labels for reconstruction. We then introduce lightweight, trainable user and item embedding modules optimized with a multi-task objective that combines next-item prediction and profile reconstruction. These modules are trained jointly to align collaborative signals with the LLM’s semantic space without modifying the backbone. We also employ token-aware loss decomposition and frequency-aware reweighting to stabilize training and mitigate popularity bias. Experiments on public benchmarks show that PALRec consistently outperforms fully fine-tuned counterparts in recommendation accuracy while preserving the LLM’s pre-trained knowledge. This result highlights that maintaining the LLM’s semantic understanding is crucial for effectively exploiting textual information in recommender systems.