Do Retrieval-Trained Embeddings Help Linear Contextual Bandits?

Abstract

Text embeddings from retrieval-tuned (dual-encoder) models are increasingly used as context features in contextual bandits for recommendation, on the assumption that an embedding space optimized for inner-product similarity will speed up a linear exploration policy. This study tests that assumption with a controlled, shared-encoder design: the same BERT-base model in two forms, vanilla (mean-pooled) and retrieval-fine-tuned (MS MARCO dot-product), used as frozen bandit context. Experiments span three datasets (MIND, MovieLens-1M, Amazon CDs and Vinyl), two linear policies (LinUCB and linear Thompson sampling), and two dimensionality-reduction methods (PCA and random projection), over 20 seeds. Neither the training objective nor the reduction method determines performance on its own; cumulative regret is governed by their interaction, which is dataset-dependent. On MovieLens the reduction reverses the encoder ranking (PCA favors the retrieval-tuned encoder, random projection the vanilla one), with large effects in both directions, while on MIND the encoders are close. The reduction method can be the larger lever, moving up to 34% of the learnable margin. A ridge-regression probe on the candidate contexts screens the reduction choice offline, without running the bandit. For linear exploration policies, the encoder and the reduction should be treated as a joint choice rather than assuming retrieval-tuned embeddings are universally preferable.