UEmbed (Unified Embedding), a decoder-only multimodal embedding model that produces both sparse lexical and dense representations in one causal forward pass, offers a new paradigm: it unifies dense and sparse embeddings in one model, while further extending sparse retrieval to unify text and multimodal inputs.
Tingyu Song, Mingxin Li, Yanzhao Zhang et al.· 2 citations
Document Embedding Preservation Tuning (DEPT) is introduced, which keeps tuned document embeddings close to cached initial embeddings while allowing retrieval gradients to pass through straight-through decoding into the generator.
Jingyuan Wang, Richong Zhang, Zhijie Nie et al.· 0 citations
This work defines model-specific Token and Phrase Representativeness Scores (TRS/PRS) to discover tokens and key phrases that appear in a document but are poorly expressed by its embedding, which improve retrieval on a constructed hard-positive benchmark and can also improve standard in-domain retrieval benchmarks.