Sparse Autoencoders as Semantic Domain Adapters for Recommender Systems

Abstract

Pretrained text embeddings enable content-based cold-item recommendation, but their domain-agnostic objectives may underrepresent distinctions that matter within a particular recommendation domain. We investigate lightweight autoencoder adapters that specialize these embeddings without retraining or finetuning the embedding model. The autoencoders need no interactional data to train; they merely use the set of items, whose embeddings they aim to reconstruct. We compared denoising, variational, and sparse autoencoder architectures across several embedding models and datasets. While all variants substantially improved over raw embeddings, sparse autoencoders (SAE) dominated in most settings. Qualitative inspection further reveals that some sparse dimensions capture coherent domain-specific concepts, suggesting SAE’s capability to meaningfully reorganize general-purpose embeddings. These preliminary findings motivate the inclusion of sparse semantic adaptation as a distinct component of recommender systems.