SMART: LLM-Augmented Hybrid Retrieval for Dynamic Product Ads
TL;DR
A lightweight quality gate identifies coverage gaps in initial keyword results, adaptively routing only the ∼ 10% of users who benefit from semantic prospecting to the LLM path, and captures the bulk of semantic prospecting gains in Relevance Score while maintaining competitive retargeting performance at a 90% reduction in LLM costs.
Abstract
Dynamic Product Ads (DPA) require retrieving relevant items from multi-million product catalogs, balancing two competing objectives: retargeting (re-surfacing known interests) and prospecting (discovering new categories). While Large Language Models (LLMs) capture semantic intent better than traditional embedding models, deploying them at scale introduces prohibitive inference costs and lexical mismatch issues. Through controlled experiments on millions of users, we demonstrate a critical retrieval decomposition: rule-generated queries excel at retargeting on a lexical BM25 index, while LLM-generated queries excel at prospecting on a dense ANN index. Building on this, we propose SMART (SeMantic-aware Adaptive ReTrieval). To manage costs, a lightweight quality gate identifies coverage gaps in initial keyword results, adaptively routing only the ∼ 10% of users who benefit from semantic prospecting to the LLM path. Offline evaluation demonstrates that this gated approach captures the bulk of semantic prospecting gains in Relevance Score while maintaining competitive retargeting performance at a 90% reduction in LLM costs. Finally, in a 2-week online A/B test at Snap, SMART improved the ad conversion rate by + 54.2% over a strong embedding-based baseline.