Aug 2026· arXiv.org· Vol abs/2608.06972· 0 citations· 29 references
Computer Science
TL;DR
The Generative Embedding Benchmark (GEB) is introduced, in which a decoder answers questions using only a frozen embedding and question text, without access to the original image or intermediate visual features, showing that generative readout exposes information bottlenecks that separability-based evaluation does not capture.
Abstract
Embeddings have emerged as a standard representational interface linking foundation models with downstream systems. Most embedding benchmarks assess representations through discriminative tasks or geometric criteria centered on separability in embedding space. However, strong performance on such evaluations does not establish whether content compressed into an embedding remains accessible to a downstream generator. To address this gap, we introduce the Generative Embedding Benchmark (GEB), in which a decoder answers questions using only a frozen embedding and question text, without access to the original image or intermediate visual features. Answer quality under this readout measures generative information: the answer-relevant content recoverable from an embedding. GEB includes a curated visual-question-answering dataset with a 1,800-item development split and a held-out 900-item test split covering natural images, scene text, and visual documents. Using a common decoder and training recipe, we evaluate seven public embedding models in visual-only and vision-language joint modes. On the test set, visual-only scores range from 28.25 to 33.21; with image-question joint encoding, all five VLM-based embedding models score higher, and the best reaches 65.56. Matched embeddings also outperform text-only inputs, zero embeddings, and shuffled embeddings. Natural-image information is much easier to recover than scene text or visual-document information, while a Qwen3-VL-2B reference with access to the original image reaches 84.30. Together, these results show that generative readout exposes information bottlenecks that separability-based evaluation does not capture.
This paper introduces a novel multimodal learning framework that directly projects visual features into linguistic embeddings, enabling robust, domain-adapted object recognition across diverse categories. Leveraging state-of-the-art visual (Detectron2 Mask R-CNN) and language (TinyLlama) models, the architecture incorp...
J. Kralev· Proceedings of the ... Inter...· 0 citations
Composed Image Retrieval (CIR) aims to retrieve a target image given a reference image and a modification text. Although recent approaches leveraging Multimodal Large Language Models (MLLMs) have shown significant progress, we argue that the MLLM's compositional understanding remains underutilized. This stems from dire...
WonJun Moon, Seongchan Kim, Dahun Kim et al.· 0 citations
CoVA is introduced, a lightweight framework that composes VAE features online into stage-wise guidance for flow-matching training, and uncovers a hidden confound in soft label conditioning: averaging class embeddings attenuates their magnitude, coupling semantic interpolation with conditioning strength.
Zhao-Wei Chen, Yi-Kai Ge, Lin-Ze Li et al.· 0 citations
Text embedding inversion reconstructs source text from fixed-dimensional vectors, posing a privacy risk to systems that store embeddings instead of raw text. We adapt the vec2text framework (Morris et al., 2023) to support text-embedding-3-small and mistral-embed alongside GTR-base. We evaluate 800 samples per category...
Kareem Elaasar, Cristina V. Lopes· Proceedings of the 9th Black...· 0 citations
Code embedding models have become a key building block for many software engineering applications, yet their internal representations remain poorly understood. Existing evaluations tend to focus on downstream task performance, offering limited insight into whether a single embedding can support both functional abstract...
Native multimodal pre-training trains a single model from scratch on text and multimodal data, enabling it to understand both language and visual inputs. However, how visual signals shape scaling behavior and support capabilities beyond visual understanding remains unclear. We investigate this question through controll...
Haoyuan Wu, Ao-Qi Wu, Jia-Ni Hu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.