2026· Poster Volume 0008 The 2026 Twenty-Second International Conference on Intelligent Computing July 23-26, 2026 Toronto, Canada· pp. 2517-2545· 0 citations
TL;DR
The Task-Adaptive Hier-archical Prompt (TAHP) framework is proposed, which guides feature extraction through dynamically generated, task-specific prompts structured at three hierarchical levels: task-type, task-content, and general prompts.
Abstract
Composed image retrieval (CIR) seeks to retrieve target images using multi-modal queries, specifically a reference image paired with modification text. Central to CIR is integrating textual semantic modifications with visual content. Despite its importance, existing approaches typically employ a static fusion paradigm, failing to account for the semantic heterogeneity of user queries, which encompass diverse task types (e.g., addition, replacement) and var-ied content. To address these limitations, we propose the Task-Adaptive Hier-archical Prompt (TAHP) framework. TAHP guides feature extraction through dynamically generated, task-specific prompts structured at three hierarchical levels: task-type, task-content, and general prompts. Furthermore, we design a Prompt Dynamic Generation Module to adaptively synthesize prompts condi-tioned on user queries and introduce a False Negative Correction Loss to optimize cross-modal feature fusion. Extensive experiments on FashionIQ and CIRR datasets demonstrate that TAHP achieves state-of-the-art performance against existing CIR approaches.
This work proposes an asymmetric fusion mechanism to generate dual retrieval queries of different granularity, enabling the model to fully use multi-modal information and introduces a bi-directional training paradigm to ensure retrieval consistency and further exploit the triplets.
A new model based on a Large Multimodal Model (LMM) that functions as a context-aware reasoner for CoCo-IR is proposed, which interprets the entire interaction history to generate Transformable Image Embeddings (TIE) that evolve across turns.
Shengcao Cao, T. Dabral, Z. Ding et al.· 0 citations
Composed Image Retrieval (CIR) is an emerging paradigm in content-based image retrieval that enables users to formulate compositional queries by combining a reference image with an auxiliary modality, usually text-based. This approach supports fine-grained search where the target image shares structural elements with the user-provided image while incorporating the modifications specified by the auxiliary text. Conventional CIR methods rely on multimodal fusion to combine visual and textual features into a joint query embedding, which requires training modules that align composed queries with the targets. In this work, we propose PeFuse (for pseudo-fusion), a training-free framework that leverages pretrained Diffusion Models and Multimodal Large Language Models to bridge modalities via generative conversion. We introduce two novel strategies: uni-directional and bi-directional conversion, which convert CIR into four single-modality retrieval problems. These methods reformulate CIR as either intra-modal or cross-modal single-query retrieval tasks, bypassing the need for dedicated task-specific training. Extensive experiments on standard benchmarks demonstrate that converting CIR into text-to-image retrieval tasks is more effective than alternative conversion strategies, achieving competitive or superior performance compared with state-of-the-art methods, while maintaining high flexibility thanks to replaceable components of the conversion pipeline. These results highlight the effectiveness of the pseudo-fusion paradigm for zero-shot CIR. Our code is publicly available at: https://github.com/StevenXuf/PeFuse4CIR.
Fan Xu, Luis A. Leiva· Trans. Mach. Learn. Res.· 0 citations
The Semantic-Aware Fine-Tuning (SAFT) framework is proposed to address semantic compression in specific domains, which incorporates Semantic-Aware Soft-Label Supervision and Intra-modal Structural Distillation to establish a promising paradigm for domain-specific TBIR tasks.
Jingyang Tan, Shengan Yang, Yuanpeng Chen et al.· 0 citations
The novel task of cross-modal query suggestion is introduced, which interactively guides users by suggesting textual refinements based on visual clusters identified in the retrieval results, and the creation of CroQS, a benchmark dataset comprising 50 diverse queries and 295 semantic clusters in generic domain.
Giacomo Pacini, Nicola Messina, Nicola Tonellotto et al.· 0 citations
Scene Text Retrieval (STR) aims to search images containing a given textual query within large-scale image collections. However, existing approaches are fundamentally constrained in two ways: 1) they are evaluated on narrow benchmarks that focus primarily on natural scenes; and 2) they rely on either error-prone multi-stage recognition-then-matching pipelines or localization-assisted matching strategies. To address these limitations, we introduce MuST, the first comprehensive benchmark for multi-scene and bilingual STR tasks, covering a broad range of real-world scenarios with carefully curated Chinese and English textual queries. On top of this benchmark, we propose BPE-Ret, a novel Byte-Pair Encoding (BPE)-level retrieval framework built on a simple yet powerful principle: a word is considered present in an image if and only if all of its constituent subwords are present. Concretely, BPE-Ret decomposes textual queries into BPE subwords and directly aligns them with dense visual features within a unified embedding space, thereby eliminating the need for explicit text spotting and coarse-grained word-level matching. We further enhance fine-grained alignment through two key innovations: a weighted preference learning scheme that prioritizes challenging cases to sharpen discrimination on confusable word-image pairs, and a subword inclusive-OR matching strategy that enforces constituent subword verification to enable robust retrieval beyond word-level granularity. Extensive experiments show that BPE-Ret establishes new state-of-the-art performance on both existing public STR benchmarks and our newly proposed MuST dataset, demonstrating the effectiveness and robustness of subword-level retrieval for real-world, multilingual scene text understanding.
Tong-Kun Guan, Yu-Tong Cai, Haocheng Wang et al.· IEEE Transactions on Image P...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
