This work proposes lightweight task-adaptive modules (TAMs), which keeps LLM parameters fixed and transforms offline embeddings into task-specialized representations without full-model backpropagation, drastically cutting training costs.
Abstract
Large language models produce semantically rich embeddings, yet direct employment of generic LLM embeddings fails to satisfy news recommendation demands due to inherent semantic mismatches with task targets. Existing fine-tuning methods including LoRA can narrow such gaps but bring prohibitive computational overhead, restricting real-world deployment. This work proposes lightweight task-adaptive modules (TAMs). It keeps LLM parameters fixed and transforms offline embeddings into task-specialized representations without full-model backpropagation, drastically cutting training costs. Evaluated on MIND benchmarks across ten mainstream recommendation architectures, TAMs achieves comparable accuracy to LoRA, with computational cost reduced to 1/10 of LoRA’s level. TAMs outperform GloVe-based models by 1.3–12.4% in AUC and scale effectively to MINDlarge. Ablation experiments confirm that the nonlinear projection is pivotal to performance improvement, and statistical validation across three random seeds confirms result robustness. This paradigm provides an efficient low-cost solution for LLM-based news recommendation under resource constraints.
Large Language Models (LLMs) have demonstrated remarkable general-purpose abilities across a wide range of domains, and these strengths have also been increasingly evidenced in recommender systems. However, existing methods that attempt to integrate collaborative signals into LLMs often fail to preserve their foundational knowledge. This loss is critical in text-rich recommendation, where robust semantic understanding is required to interpret user reviews and item profiles. We propose PALRec, a parameter-preserving augmentation framework that equips an LLM with recommendation capabilities while keeping its original parameters fixed. We first construct evidence-grounded user and item profiles from reviews and use them as concise pseudo-labels for reconstruction. We then introduce lightweight, trainable user and item embedding modules optimized with a multi-task objective that combines next-item prediction and profile reconstruction. These modules are trained jointly to align collaborative signals with the LLM’s semantic space without modifying the backbone. We also employ token-aware loss decomposition and frequency-aware reweighting to stabilize training and mitigate popularity bias. Experiments on public benchmarks show that PALRec consistently outperforms fully fine-tuned counterparts in recommendation accuracy while preserving the LLM’s pre-trained knowledge. This result highlights that maintaining the LLM’s semantic understanding is crucial for effectively exploiting textual information in recommender systems.
Hyunsoo Na, Minseok Gang, Sang-goo Lee et al.· ACM Transactions on Informat...· 0 citations
The Fusion of Layer-wise Exits for Sequential Recommendation (FLEXRec), a discriminative framework that enhances compact LLMs while retaining scalable full-corpus ranking and achieves state-of-the-art accuracy among competing methods while remaining highly efficient.
Xurong Liang, Tong Chen, Q. Nguyen et al.· 0 citations
REPREC is a lightweight framework that conditions a frozen LLM using compact user-level representations and consistently improves recommendation performance across different sequential encoders, LLM backbones, and user activity levels.
Large language models (LLMs) have shown promise for training-free recommendation, but LLM-generated user interests are often too broad for fine-grained item retrieval. Existing methods incorporate collaborative filtering (CF) signals in a pre-LLM manner through candidate reranking or prompt augmentation, yielding limited gains. We propose CoRRe, a training-free recommendation framework with a post-LLM paradigm that injects CF signals into LLM-generated item representations, which are later matched with LLM-generated user interests for ranking. Specifically, CoRRe refines the directions of item embeddings using an item-item co-purchase graph and their magnitudes using item popularity. Experiments on real-world datasets show that CoRRe consistently outperforms existing training-free methods and achieves competitive or superior performance compared with training-based methods, without requiring any model training or task-specific fine-tuning.
Kyungho Kim, Sunwoo Kim, Geon Lee et al.· 0 citations
This work proposes eXtreme Gradient Boosting LoRA (XGBLoRA), a novel framework grounded in gradient boosting theory that provides theoretical analysis establishing convergence guarantees and expressiveness bounds, which formally justify why weaker (lower-rank) adapters, when properly combined, can match or exceed the performance of stronger ones.
Yifei Zhang, Hao Zhu, Haoran Shi et al.· Proceedings of the 32nd ACM...· 0 citations
This work proposes a simple, parameter-free intervention that initializes SID token embeddings directly from their corresponding centroids in the semantic embedding space, and shows that preserving SID geometry, beyond shared-prefix structure, provides a simple and effective semantic prior for LLM-based GR.
Donald Loveland, Liam Collins, B. Kumar et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
