Skip to content
Preprint

Query Independent Variable Rate Visual Token Coding

Sep 2026 · 0 citations · 14 references
Computer Science

TL;DR

This work takes the other half of the classical transform-coding toolkit instead: keep every token and vary its rate, and preserves the model's output distribution and its answers better than uniform-rate coding, the closed-form water-fill and distortion-ranked pruning.

Abstract

Visual-token compression for vision--language models is posed almost entirely as a selection problem: decide which tokens to keep and discard the rest. The criteria that work best rank tokens by the attention the language model pays them, which makes the ranking a function of the question being asked. That is invisible in a single-turn benchmark and decisive whenever a compressed representation is written once and read many times, as when it is cached across the turns of a conversation or transmitted between a device and a server. We take the other half of the classical transform-coding toolkit instead: keep every token and vary its rate. A transform code exposes each token's measured distortion--rate curve, and a fixed bit budget is distributed across tokens by exact integer rate--distortion optimisation on those curves. No text enters the pipeline, so one compressed representation serves any query. At equal bit budgets, on two datasets and two capacities, it preserves the model's output distribution and its answers better than uniform-rate coding, the closed-form water-fill and distortion-ranked pruning. It matches attention-ranked pruning on the question pruning was tuned for, and overtakes it once the compressed image must answer a different question about the same image.

View source

Similar papers

Preprint Oct 2026

Query-adaptive Recurrent Tokenization

Visual information worth extracting from an image and the token budget it warrants depend on the text query. Yet modern vision and vision-language models largely encode images independently of the query and allocate a fixed visual token budget to every input. We introduce the Query-Adaptive Recurrent Tokenizer (QART),...

Bavesh Balaji, P. Gopinath, Pouya Sadeghi et al. · 0 citations
#small language model Preprint Sep 2026

Compressing Streaming Neural Audio Encoders via Latent-Space Distillation

This work studies how to compress a tokenizer by distillation, taking as the supervision target neither the discrete token nor the output distribution but the pre-quantizer latent the model actually consumes - the last representation the two token interfaces share.

Prasanth Yadla, Mohammad Samragh, Dongseong Hwang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval

This work introduces Matryoshka Hash Representations (MHR), a two-stage procedure that separates full-width training from prefix organization and strengthens two common pipelines: shortlisting candidates for full-precision reranking, and pruning a low-storage graph index such as LEANN.

Peichun Hua, Yun-Ming Xiao · 1 citation

CoViST: Visual Token Compression via Composable States

CoViST is proposed, a training-free framework that represents a compressed image as a composable visual state that retains its interpretation under successive reductions, enabling the same formulation to support both fixed compression before prefill and progressive compression within the decoder.

Qi Zhang, Xian-Dong Meng, Rong-Gang Wang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

VETO: Video Efficient Token Optimization for Vision Language Models

The key design insight is hierarchical ordering: by first compressing spatial dimensions, VETO drastically reduces the cost of subsequent global temporal matching, bypassing the efficiency wall of single-axis approaches, with an advantage that grows with modern fully-fused attention infrastructure.

Gueter Josmy Faure, Hao-Ping Wang, Min-Hung Chen et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.