Skip to content

Query Suggestion for Text-to-Image Retrieval

Nov 2026 · ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) · 0 citations · 38 references

TL;DR

Cross-modal Query Suggestion, the task of dynamically generating targeted query alternatives that verbalize the semantic content of visually coherent image groups within an initial retrieval result set, is investigated, and a comprehensive evaluation framework to map the structural trade-offs is established.

Abstract

The challenge of translating a user's information need into an effective search query is fundamental in modern cross-modal retrieval systems, which allow users to search image collections with natural language but often lack interactive refinement tools. Our paper addresses the gap by investigating Cross-modal Query Suggestion, the task of dynamically generating targeted query alternatives that verbalize the semantic content of visually coherent image groups within an initial retrieval result set. Building upon our prior work establishing a dedicated benchmark, this extended study systematically compares three distinct methodological paradigms for generating these suggestions: i) latent-conditioned generation, which operates on visual embeddings; ii) caption-based generation, relying on textual abstractions of the result set; and iii) image-conditioned generation, which directly utilizes representative images from the result set as input for Large Multimodal Models (LMMs). We further expand the evaluation framework by introducing new metrics for semantic fidelity against human annotations and computational efficiency. While our evaluation assumes a controlled setup with disjoint image clusters, we validate its robustness against automated partitioning and noise. This work establishes a comprehensive evaluation framework to map the structural trade-offs of these paradigms, and validates the effectiveness of the proposed protocol with a tailored user preference study. Our experiments demonstrate that approaches utilizing direct visual grounding via LMMs produce more image-specific suggestions, Large Language Models (LLMs) reach the best tradeoff between the assessed properties, while latent-conditioned methods have the lowest computational cost. These findings offer critical insights for developing dynamic, visually informed interactive search systems. The dataset, the methods implementations and the code to replicate our experiments are available at: github.com/Ruggero1912/CroQS-benchmark

View source

Similar papers

#natural language process... Preprint Oct 2026

Cascaded Retrieval with Cross-Modal Query Refinement for Efficient Visual Document Search

This work proposes SCOUT, a cascaded two-track retrieval framework that separates retrieval performance from exhaustive search, and evaluates it on 4 major benchmarks against 6 baselines and observes a significant query-time latency improvement up to 80.7x.

Diana Turmakhan, Aseem Srivastava, Vivek Gupta et al. · 0 citations
Preprint Oct 2026

EVIE: Evidence-Vector-Informed Embeddings for Visual Document Retrieval

This work introduces EVIE (Evidence-Vector-Informed Embeddings), a family of native visual document retrievers integrating three key innovations: evidence-judged data governance, which uses a multimodal judge to identify answer-bearing positives and filter unreliable negatives, and Hierarchical agglomerative index comp...

Zi-Fei Wang, Wei Wen, Qian Ji et al. · 1 citation
#artificial intelligence Review Sep 2026

Graded-Relevance Composed Multimodal Retrieval for E-commerce Visual Search at Scale

Visual search on large e-commerce catalogs must serve both"similarity"queries that ask for items resembling an uploaded image and"modifier"queries that comprise an image and text describing a desired modification (e.g. a color change or style swap). The latter is the setting known as composed image retrieval (CIR). Exi...

Anubhav Gupta, Hrushikesh Mohapatra, Prijith Chandra et al. · 0 citations
Open access Nov 2026

A Retrieval-Augmented Conversational Framework for Intelligent Personal Document Understanding

The exponential proliferation of digital content across personal repositories—encompassing academic manuscripts, annotated lecture transcripts, technical project documentation, and domain-specific study materials—has introduced substantial challenges in efficient knowledge retrieval. Conventional information retrieval...

Vaibhav Sharma · 0 citations
Review Oct 2026

Pix2Evidence: A Multimodal RAG System for Text and Visual Evidence

Retrieval-augmented generation (RAG) over visually rich documents must find both relevant content and the evidence that supports an answer. Screenshot retrieval preserves images and page structure, but written facts within screenshots also need to be searchable. Moreover, topical relevance alone does not identify which...

Zi-Niu Liu, Xiaoyang Chen, De-Zhi Ye et al. · 0 citations
Book Open access Nov 2026

From Dialogue to Evidence: Retrieval-State-Conditioned Interaction for Text-Based Person Retrieval

Interactive text-based person retrieval aims to resolve ambiguity in an underspecified initial query through iterative clarification. However, existing methods largely rely on free-form dialogue, which makes interaction depend on unstructured language generation rather than directly selecting the most discriminative vi...

Bao Tran, Thanh Duc Ngo · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.