Aug 2026· IEEE Transactions on Image Processing· Vol 35, pp. 8952-8965· 0 citations· 63 references
MedicineComputer Science
Abstract
Video Corpus Moment Retrieval (VCMR) is pivotal to multimodal understanding. However, existing methods rely heavily on large-scale annotated data, which limits their generalization and scalability. To address this issue, we propose a training-free VCMR framework, termed Synergistic Collaboration and Adaptive Calibration (SCAC), enabling effective semantic parsing and precise temporal localization without parameter updates. SCAC introduces a Query Event Chain Generation module that leverages large language models to transform complex textual queries into structured event chains, while a Video Event Chain Generation module represents videos as semantically coherent event chains through subtitle segmentation and keyframe aggregation. Built on these structured representations, SCAC performs Event-Chain-Based Cross-Modal Retrieval with mean-variance joint scoring to suppress local mismatches and reinforce global consistency. During localization, a Synergy-Calibration Mechanism dynamically refines temporal boundaries via profit-setback feedback. Extensive experiments show that SCAC achieves comparable or superior results to supervised counterparts under training-free conditions, demonstrating strong cross-modal generalization and adaptive capability. The code of our method is available at https://github.com/cyanlll/SCAC
This work proposes Video-GAR, a novel framework that reframes the conventional retrieval task from superficial matching to generative understanding, positing that the capability for query reconstruction evidences deep semantic comprehension.
Mingjin Kuai, Qianyin Xiao, Juncheng Li et al.· Annual International ACM SIG...· 0 citations
Empirical evaluations across a diverse suite of multimodal document retrieval benchmarks reveal that DART achieves consistent and significant gains in ranking precision, and this dynamic refinement process introduces minimal computational latency, offering a highly efficient, plug-and-play solution for adaptive document retrieval.
Jing Zhang, Yaowei Wang, Chongyu Wang et al.· International Conference on...· 0 citations
This work proposes Self-SiMS, a self-similarity-based Moment Proposal and Scoring that exploits intrinsic relationships within videos, enabling robust span generation and scoring and introduces a query-aware MLLM-based reasoning stage to further sharpen alignment between text and video.
Jihyun Lee, Cheol-Ho Cho, Woojin Jun et al.· arXiv.org· 0 citations
A structured, comprehensive survey of the latest MVU progress is presented, establishing a novel three-tier taxonomy that categorizes existing studies into cross-modal alignment, multi-granularity semantic expression and multimodal reasoning.
Rongyong Zhao, Da Pu, Cuiling Li et al.· Applied Sciences· 0 citations
A new prototype-based hierarchical alignment network (PHA-Net) to align individual/local/global level representations across modalities and introduces multiple modality-shared prototypes as the bridge to efficiently optimize text and video representations for cross-modal alignment.
Xiaolun Jing, Kezhao Yin, Xinxing Yang et al.· Neurocomputing· 0 citations
Retrieval-Augmented Generation (RAG) has established itself as a compelling strategy for grounding large language model outputs in documentary evidence. However, production deployments continue to rely almost exclusively on homogeneous text corpora, even as enterprise repositories grow increasingly heterogeneous blending technical schematics, radiological images, annotated diagrams, and unstructured prose within the same archival system. This mismatch between system design and data reality motivates the present work. We propose a Multimodal RAG framework that unifies text and image retrieval through four tightly coupled components: a dual-stream embedding engine, a learned four-class AI query router, an adaptive confidence threshold, and a session-aware context store. Text is encoded with the allmpnet-base-v2 Sentence Transformer; images are embedded in the same 768-dimensional space via a domain-adapted CLIP ViT-L/14 model fine-tuned on approximately 120,000 technical and clinical text-image pairs. Concatenating 768-d vectors yields a 1,536-d composite query that drives a single approximate nearest-neighbor (ANN) search simultaneously across both modalities. A fine-tuned DistilBERT router assigns each query to one of four retrieval pathways text-only, image-only, hybrid, or conversational at 91.3% accuracy. Experiments were run on three enterprise corpora (engineering manuals, clinical case summaries, and legal paperwork) with the results being a mean F1@5 of 0.90, an increase of 22% over a dense text-only baseline, and a reduction of 49% in the number of hallucinations. The median first-token latency is 1.34s, which meets the interactive-use target deployment-contexts. These results show that it is possible to implement modality-aware retrieval in a simple and realistic setting with real-world organizational constraints and that the results are reproducible.
E.Vijayakumar, G. A.· 2026 4th International Confe...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
