Skip to content
Preprint

Tracing Query Expansion Effects through Sparse Autoencoder Features

Sep 2026 · 0 citations · 47 references
Computer Science

TL;DR

Analysis shows that effective QE induces layer-concentrated changes in sparse latents aligned with retrieval intent and entity attributes, rather than only perturbing final query embeddings, and suggests that SAEs can explain QE effects and offer a lightweight option for precise retrieval behavior modulation without query rewriting or retriever fine-tuning.

Abstract

Query expansion (QE) is a critical technique in information retrieval that enriches underspecified queries with additional textual context. However, its effect is often unreliable in modern dense retrieval, especially for strong off-the-shelf retrievers without retraining. Existing studies mainly examine expansion quality, semantic drift, or retrieval outcomes, but rarely explain how QE changes dense retrievers internally. In this work, we trace QE effects through sparse autoencoder (SAE) features. Using paired original and expanded queries, we decompose layer-wise retriever representations into sparse latent activations, identify QE-related latents from expansion-induced activation shifts, and interpret them with natural-language descriptions and retrieval cases. Our analysis shows that effective QE induces layer-concentrated changes in sparse latents aligned with retrieval intent and entity attributes, rather than only perturbing final query embeddings. SAE-based activation steering further validates these latents improve retrieval more consistently than random interventions or vanilla QE across four benchmarks, suggesting that SAEs can explain QE effects and offer a lightweight option for precise retrieval behavior modulation without query rewriting or retriever fine-tuning.

View source

Similar papers

#small language model Preprint Aug 2026

Query Expansion Is More Than Generation: Improving Dense Retrieval through Better Integration

This work introduces AnchorQE, a training-free method that separately encodes the original query and its expansion before interpolating them, and shows that AnchorQE improves retrieval effectiveness by up to 12.89% when compared to widely-used expansion-only or text-level concatenation baselines across TREC-DL, LoTTE,...

Sixia Sun, Mihai Surdeanu · 1 citation
#natural language process... Preprint Oct 2026

Finding the Right Balance: Relevance and Diversity in LLM Retrieval

It is concluded that diversification should be used selectively, based on observable redundancy and evidence requirements, and introduces a query-adaptive rule that diversifies only when the effective number of distinct documents in the nearest-neighbor top-$k$ selection falls below the query's evidence requirement.

Guillaume Brouillette, Faustin Kagabo, Usef Faghihi et al. · 0 citations
Open access Oct 2026

Query Expansion Using Wikidata and Hybrid Dense-Sparse Retrieval for Semantic Information Access

Standard information retrieval systems often miss deeper meaning when queries are short, vague, or lack specifics. We propose a hybrid framework that pulls structured data from Wikidata and combines dense and sparse retrieval methods to improve both finding documents and generating answers. Our method first identifies...

Abhishek Dwivedi, Shesh Mani Tiwari, Mohd. Shah Alam et al. · 0 citations
#small language model Preprint Oct 2026

XViewRank: Overlap-Conditioned Score Fusion for LLM-Augmented Sparse Retrieval

XViewRank is presented, a lightweight retrieval-side method for conditionally fusing BM25 results from an original query, a hypothesis view, and a step-back view, and the results support overlap-conditioned score fusion as a compact retrieval-side alternative to fixed fusion, while also delimiting where its advantage i...

Xin-Xuan Lv, Ming-Han Li, Guo-Dong Zhou · 0 citations
Preprint Aug 2026

DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval

Document Embedding Preservation Tuning (DEPT) is introduced, which keeps tuned document embeddings close to cached initial embeddings while allowing retrieval gradients to pass through straight-through decoding into the generator.

Jingyuan Wang, Richong Zhang, Zhijie Nie et al. · 0 citations
Sep 2026

Test-Time Adaptation with Query-Dependent Residuals for Visual Document Retrieval

Visual document retrieval (VDR) systems depend on page embeddings computed before deployment, which makes adaptation difficult when encoder parameters or corpus re-encoding are unavailable. Rerankers provide useful relevance signals, but conventional reranking applies them only to selected queries and candidate pages....

Ze-Liang Li, Xiao-Fen Xing, K. Guo et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.