GradSup: Gradient Superposition for Personalised and Scalable LLM Recommendation
TL;DR
Experiments conducted on movie, book and electronics benchmark datasets demonstrate that GradSup outperforms iterative fine-tuning to provide scalable and personalised recommendation that is consistently sustained above the frozen LLM backbone.
Abstract
Large language models (LLMs) have demonstrated strong capabilities in recommendation tasks such as item, sequence, conversational recommendation, and explanation generation. However, LLM weights are typically shared across all users. Adapting these models to individual users remains a fundamental challenge that requires millions of trainable parameters, even when using finetuning methods such as Low-Rank Adaptation (LoRA). Building such personalised adapters would also require large volumes of storage and high computational overhead. To address this challenge of personalised and scalable recommendation, we propose the Gradient Superposition (GradSup) method. Built on the TinyLoRA architecture, GradSup is a closed-form method that computes per-user adapters in a single pass, achieving 15 × speedup over iterative fine-tuning at matched parameter budget and 400 × compression over full LoRA. Experiments conducted on movie, book and electronics benchmark datasets demonstrate that GradSup outperforms iterative fine-tuning to provide scalable and personalised recommendation that is consistently sustained above the frozen LLM backbone. Code is available at: https://github.com/kanishkaRandunu/GradSup