TL;DR
The proposed FastVSearch method bridges efficient data indexing and deep multimodal understanding, enabling practical deployment for large-scale video search applications.
The explosive growth of large-scale video archives from surveillance networks, online platforms, and personal devices has made efficient and semantically rich video retrieval a critical challenge. Existing approaches based on deep multimodal embeddings have significantly improved retrieval accuracy. However, they often...
Modern vision-language models (VLMs) have shown promising results in long-video understanding due to the rich semantic information they can capture. However, most methods focus on coarse captioning of extracted image frames that are computationally inefficient and require models with large context windows. While past w...
Scene Text Retrieval (STR) aims to search images containing a given textual query within large-scale image collections. However, existing approaches are fundamentally constrained in two ways: 1) they are evaluated on narrow benchmarks that focus primarily on natural scenes; and 2) they rely on either error-prone multi-...
Streaming video understanding requires Vision Language Models (VLLMs) to process growing video streams and answer user questions under tight latency constraints. Existing methods improve efficiency through token pruning and memory-bank schemes, but mainly reduce visual tokens after visual encoding. Consequently, downst...
Large-scale image retrieval requires compact representations without substantially sacrificing retrieval accuracy. However, Vision Transformer Hashing (VTS) concatenates all output tokens before hash projection, resulting in a high-dimensional hashing head with considerable model and memory overhead. We replace this to...
Ad-hoc video search retrieves relevant video shots from large-scale collections using free-form textual queries. Pretrained vision-language models provide strong frame-level matching, but max-similarity aggregation can miss semantic evidence distributed across neighboring frames. We propose a lightweight retrieve-then-...
StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
