Aug 2026· Engineering, Technology & Applied Science Research· Vol 16, pp. 37231-37238· 0 citations· 17 references
TL;DR
It is established that document selection, rather than term weighting, is the primary lever for PRF improvement and the vocabulary gap is identified as the principal target for next-generation methods.
Abstract
Pseudo-Relevance Feedback (PRF) has remained a cornerstone of unsupervised retrieval since Rocchio (1971), yet the foundational assumption that the top- retrieved documents are the best available feedback has received limited direct empirical scrutiny, despite widespread adoption in both classical and neural approaches. This study analyzed BM25 retrieval on the TREC Deep Learning 2019 and 2020 test collections (97 queries, 20,646 graded relevance judgements) and found that 79% of relevant documents fall outside the top-10, with a mean rank of 37.3. An oracle selection strategy achieved 0.324 higher feedback precision at , defined as the proportion of graded-relevant documents within the selected for expansion, with a large effect size that is consistent across all tested values of and both test collections. LLM-based analysis of 61 extreme cases identified vocabulary gap as the dominant failure mode in 96.7% of cases, driven primarily by implicit relevance (35.6%) and hypernym-hyponym mismatch (27.1%). These findings establish that document selection, rather than term weighting, is the primary lever for PRF improvement and identify the vocabulary gap as the principal target for next-generation methods. The results demonstrate that improving feedback-document selection represents a largely unexplored avenue for PRF advancement.
seek, Self-Evaluative Exploration for Knowledge Retrieval, a training-free framework that addresses this limitation through iterative corpus interaction at test time through iterative corpus interaction at test time.
Amin Bigdeli, Radin Hamidi Rad, Negar Arabzadeh et al.· Proceedings of the 35th ACM...· 1 citation
New approaches based on the recently proposed Dimension IMportance Estimators (DIMEs) are introduced, incorporating LLM-generated relevance judgments into the re-ranking process, showing that even a few noisy relevance judgments from small LLMs can provide a powerful signal to improve the effectiveness of IR systems.
Andrea Pasin, Guglielmo Faggioli, Nicola Ferro et al.· Proceedings of the 35th ACM...· 0 citations
Large language models (LLMs) are increasingly used to produce the relevance labels that underpin the evaluation of information retrieval (IR) and retrieval-augmented generation systems. Yet delegating relevance assessment to an LLM raises a meta-evaluation question that is hard to answer with existing resources: when,...
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi et al.· Proceedings of the 35th ACM...· 0 citations
It is concluded that diversification should be used selectively, based on observable redundancy and evidence requirements, and introduces a query-adaptive rule that diversifies only when the effective number of distinct documents in the nearest-neighbor top-$k$ selection falls below the query's evidence requirement.
Guillaume Brouillette, Faustin Kagabo, Usef Faghihi et al.· 0 citations
Retrieval-based in-context learning (ICL) enables large language models (LLMs) to predict tabular labels from similar examples, but it remains unclear whether LLM inference improves on direct voting over those same examples. We address this question by comparing four general-purpose LLMs with distance-weighted nearest-...
Zi-Quan Wang, Yu-Yang Nie, Jian Ma et al.· 0 citations
Hybrid retrieval, which fuses a sparse lexical retriever with a dense neural one, is widely recommended in the software documentation surveyed in September 2026, usually with reciprocal rank fusion (RRF), on the strength of aggregate gains. We ask what fusion costs on individual queries, treating “do not fuse” as a fir...
A. Chauhan, Arun Agarwal· IEEE Access· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.