ProRetrieval: Learning to Orchestrate Hybrid Search via Executable Program Synthesis

MetaStrategy: Generative Ranking with Executable LLM Strategies

The method combines selection, relative-rank, and baseline-lift rewards, a self-competitive curriculum that feeds frequent strategies back as competitors, and Evaluator-routed reward-augmented on-policy distillation that transfers complementary 4B-parameter Teachers into a compact 0.8B-parameter Student.

From Retrieval to Reasoning: Building Language-Steerable Discovery Systems at Spotify

For decades, search and recommendation systems have been optimized as distinct components within large-scale discovery platforms. The rise of generative AI is beginning to blur this boundary. At Spotify, we are exploring how large language models can evolve from tools that retrieve content into systems that reason over users, catalogs, and intent, while remaining steerable through natural language and user interaction. This talk presents lessons from deploying and studying generative retrieval and recommendation systems across Spotify's content ecosystem. I will describe how semantic identifiers enable language models to operate directly over large, heterogeneous catalogs, allowing search, recommendation, retrieval, explanation, and user understanding to be expressed within a common generative framework. I will discuss recent work on production-scale podcast discovery, language-steerable recommendation, and the NEO framework for unifying search, recommendation, and reasoning across multiple content types. These systems demonstrate how grounding language models in catalog entities and user behavior can improve discovery while preserving the flexibility of natural-language interaction. More broadly, they suggest a path toward discovery systems in which retrieval, recommendation, and reasoning are no longer separate stages, but capabilities of a shared generative model. Beyond model frameworks, I will discuss the emerging challenges of alignment and evaluation in discovery systems. Unlike traditional retrieval problems, generative recommendation often has many valid answers. I will present approaches for learning from large-scale behavioral signals, preference-aware optimization, and profile-aware LLM-as-a-judge evaluation, along with lessons from online experimentation at Spotify. These experiences suggest that future discovery systems will require new forms of personalization, controllability, and evaluation that extend beyond conventional ranking metrics. I will conclude with a research agenda for generative discovery systems, including language-steerable interfaces, unified retrieval-and-reasoning models, preference-aligned generation, and evaluation frameworks designed to measure user-specific relevance at scale. As search, recommendation, and conversational AI continue to converge, these directions point toward a new generation of discovery systems that can understand intent, reason over large catalogs, and help users navigate increasingly complex information spaces.

Advanced Natural Language to SQL Query Conversion: A Hybrid Approach Leveraging Deep Learning and Semantic Analysis

Harness-G: A Graph-Structured Harness for Search Agents

Harness-G, a graph-structured retrieval framework that reformulates free-form query generation as finite action selection, and introduces Structured Non-myopic Credit (SNC), which uses a frozen answer scorer to compare the selected action with its alternatives and assigns downstream gains to the earlier actions that enabled them.

Evolving Executable Pipeline Programs for AutoML with Language Models

This work presents LACE, an AutoML framework that instead searches over complete executable pipeline programs: an evolutionary loop maintains a population of scikit-learn-compatible Python classes, and a large language model acts as the variation operator.

Semantic Query Plan Optimization for AI-Powered Analytics

It is argued that the next generation of systems for semantic query optimization must move from optimized execution of fixed semantic plans toward cost-aware execution of agentic analytics workflows.