Large Language Models (LLMs) have emerged as powerful assets for recommender systems. However, deploying them as generative recommenders or zero-shot rankers at web-scale remains bottlenecked by prohibitive computational overhead and grounding challenges. In this paper, we revitalize the classic, highly efficient two-tower retrieval architecture by adapting LLMs as semantic representation backbones rather than generative engines. We introduce an LLM-native two-tower framework engineered for high-throughput, large-scale retrieval. Our architecture introduces several key innovations: a shared LLM encoder for joint user-item modeling, End-Of-Sentence (EOS) token pooling for compact sequence embedding, cross-dataset transfer learning, knowledge distillation from powerful cross-encoder teachers, and latent reasoning within the user tower. Extensive evaluation across three public benchmarks demonstrates that cross-encoder architecture outperforms current state-of-the-art (SoTA) models, while the efficient two-tower student achieves SoTA-comparable retrieval performance. Furthermore, experiments on internal large-scale production systems yield substantial topline retrieval improvements along with high resilience to model staleness and superior data scaling. Our findings demonstrate that when augmented with modern representation learning, the traditional two-tower paradigm remains an exceptionally competitive and practical solution for industrial retrieval systems.
Pretrained large language models (LLMs) are promising retrieval engines because they combine rich semantic priors, strong sequence modeling capabilities, and favorable scaling behavior. However, turning a pretrained LLM into a generative retriever in production deployment raises several challenges: the model must learn an internal item vocabulary that was absent from pretraining, and generate valid item identifiers under strict latency and cost constraints. We address these challenges through the design and launch of SnapLGR, an LLM-based generative retrieval system for short-video recommendation at Snapchat. The system is built around three main designs. First, we construct semantic identifiers (SIDs) from multimodal item embeddings and enhance them with Personalized PageRank (PPR)-based co-engagement contrastive learning, resulting in improved codebook utilization, reduced collisions, and infused collaborative signal. Second, we use continued pretraining (CPT) to ground the introduced SID tokens before supervised fine-tuning (SFT) on user interaction sequences. Third, we make SnapLGR serving practical through TensorRT-LLM CUDA-backed beam search and a decentralized worker-loop architecture. In a live A/B test, the launched system increased View Time by 0.37%, Time Spent by 0.09%, Deep Sessions by 0.18%, and Deep Sessions Unique User by 0.11% relative to the existing TIGER-style generative retrieval baseline. We then decompose this offline gap under a fixed tokenizer and quantify the gains due to model architecture, scaling, and pretraining. Overall, our deployment shows that successful production SnapLGR requires joint design across representation learning, vocabulary grounding, and efficient training and serving.
Liam Collins, Jiwen Ren, Donald Loveland et al.· 0 citations
Large embedding models improve retrieval quality, but serving large encoders online is expensive. We study whether a compact retriever can learn teacher ranking behavior from score vectors without access to teacher hidden states. The student trains on rows built from ground-truth positives and negative candidates produced by our data generation pipeline; we evaluate student-teacher hard-negative mining separately as an extension. We use a row-centered score-vector objective, a memory-efficient implementation of uniform all-pairs PairMSE loss. On a fixed eight-task evaluation panel, our distillation protocol recovers up to 50% of the base-to-teacher gap. The distilled 0.6B student is 4.7 times faster for query encoding and 9.7 times faster for document encoding than sequential online teacher fusion. External-transfer performance after distillation remains mixed, so our evidence supports compression of teacher rankings under matched retrieval protocols.
K. Dubovikov, Martin Takác, S. Lahlou· 0 citations
Generative retrieval is increasingly popular in large-scale recommendation and advertising systems, yet current methods introduce practical complications. Semantic-ID methods rely on quantization, mutable identifier vocabularies, and token-to-item grounding; embedding-based pipelines train the item encoder separately from the query generator, which limits user-item alignment. We propose EGR, an Embedding-Native Generative Retrieval framework for recommendation and advertising. EGR uses a single shared LLM to learn item representations from item metadata and user representations from interaction histories in one embedding space. Items are indexed directly as dense vectors, and user histories are encoded as dense retrieval queries. Joint contrastive training groups related items and aligns queries with their target items. We evaluate EGR on public benchmarks, industrial data, and live deployment. EGR outperforms published baselines on Amazon Reviews; on Snap DPA, it scales with data, handles cold-start items, and benefits from multimodal input. In production, EGR delivers a +2.91% conversion-rate lift, simplifying system design while improving retrieval quality and ad performance.
Xiaodong Liu, Congfei Zhang, Hsiang-wei Chao et al.· 0 citations
In recent years, large language models (LLMs) have achieved remarkable advances in code generation. However, their massive parameter scales hinder deployment in resource-constrained environments. Knowledge distillation has emerged as an effective compression technique that transfers knowledge from a large teacher model to a smaller student model, thereby reducing computational cost while retaining strong generative capability. However, traditional distillation methods usually depend on forward and reverse Kullback-Leibler (KL) divergence, aligning the probability distribution over the entire vocabulary. This process makes them susceptible to long-tail noise and often leads to weaker performance than supervised fine-tuning with labeled data. To address this issue, we propose a distillation approach based on ranking supervision. At each step, the method selects the candidate tokens with the highest probabilities from the teacher's output and applies a ListNet-based loss. This loss encourages the student to learn the teacher's ranking preferences. Unlike conventional KL distillation, ranking distillation avoids exhaustive alignment of low-confidence tokens, achieving comparable training time while significantly reducing GPU memory consumption. We conduct systematic evaluations on four public benchmarks (HumanEval, MBPP, DS-1000, and MultiPL-E). Experimental results demonstrate that the proposed method consistently outperforms supervised fine-tuning as well as FKL and RKL baselines in Python code generation, multilingual generation, and data-science scenarios. Moreover, it maintains stable performance gains across different model scales, including both the Qwen2.5-Coder and DeepSeek-Coder families. Our method provides an effective solution for distilling large language models in code generation and offers guidance for future research in model compression.
Zhe Ding, Hui Ji, Su Pan et al.· Neural Networks· 0 citations
Large language models (LLMs) have changed enterprise knowledge work. Their value, however, is capped by three failures: they hallucinate, their parametric memory is frozen and grows stale, and they cannot read the proprietary data that holds most business answers. Retrieval-Augmented Generation (RAG) targets all three. It grounds generation in passages fetched at inference time from an external, continuously updatable corpus, so answers become verifiable and citation-backed without any model retraining. This paper presents a technical synthesis of RAG for trustworthy enterprise assistants. The end-to-end pipeline is described in full: document chunking, embedding, vector indexing, retrieval, cross-encoder re-ranking, and grounded generation with inline citations. Advanced variants are then surveyed, namely hybrid sparse-dense retrieval, Hypothetical Document Embeddings (HyDE), graph-based RAG, and agentic iterative retrieval. A RAGAS-style evaluation method quantifies faithfulness, answer relevance, and context precision and recall. On an illustrative enterprise question-answering scenario, an advanced configuration that combines hybrid retrieval with cross-encoder re-ranking lifts faithfulness from 0.71 to 0.91 and context precision from 0.62 to 0.84 over naive dense-only RAG. Agentic retrieval reaches 0.95 faithfulness, but pays for it in latency. Enterprise concerns, including document-level access control, data security, cost, and latency budgets, are treated as first-class design constraints. The reported metrics are illustrative. They characterise representative trade-offs rather than a specific deployed study.
Bini P B· International Journal of Inf...· 0 citations
Recently, the generative retrieval paradigm has emerged as a transformative framework that significantly enhances the efficiency of large-scale industrial recommendation systems. This innovative approach systematically maps items to meaningful semantic identifiers (SIDs) and employs advanced sequence generation techniques to construct high-quality candidate sets, thereby enabling more accurate modeling of users' evolving interests and behavioral patterns. Nevertheless, two critical challenges remain inadequately addressed in current research: (1) Existing methodologies predominantly focus on modeling a single task such as predicting users' click behavior, while overlooking other tasks including predicting users' dwell-time and engagement behaviors, which are very important for video/content recommendation at the same time. The independent modeling of each task inevitably results in substantial computational overhead, thereby raising the pivotal question of whether the sophisticated multi-task learning capabilities inherent in LLMs can be effectively leveraged to achieve unified and efficient multi-task learning for generative retrieval. (2) The mapping mechanism from SIDs to concrete items requires substantial refinement to ensure precise and reliable retrieval performance. To tackle these issues, we propose RedGR, a generative retrieval model that unifies the modeling of multiple complex retrieval tasks. RedGR first applies the RQ-Kmeans algorithm to map items into SIDs, and then conducts pre-training on large-scale user behavior datasets to learn general knowledge. Then the RedGR model is finetuned on multi-task retrieval data with a unique instruction prompt for each task. This enables RedGR to generate the corresponding set of SIDs for each task. And the union of all sets of SIDs is the multi-task retrieval result. Finally, the Swing algorithm incorporates explicit, high-quality collaborative signals to strengthen the mapping from SIDs to specific items, thereby facilitating efficient retrieval of high-quality items. RedGR has been fully depolyed in the homefeed recommendation scenario of RedNote,serving hundreds of millions of users every day. Online A/B test results show a 0.178% increase in pagetime, a 0.734% increase in average user engagement, and a 0.076% growth in homefeed active users (FAU). These metrics collectively validate the superior performance of RedGR's unified retrieval modeling approach in complex multi-task scenarios.
Mengcheng Fang, Hongyu Wang, Xichuan Niu et al.· Annual International ACM SIG...· 0 citations