LSRTopic: Contextual Sparse Topic Modeling via Corpus-Vocabulary Alignment
Abstract
Topic modeling still struggles to combine contextual expressiveness with lexical interpretability. Sparse models are transparent but semantically limited, whereas dense contextual models capture meaning at the cost of direct interpretability. We argue that Learned Sparse Representations (LSR) offer a promising alternative but remain underused in topic modeling due to a vocabulary misalignment problem: their activations are over subword vocabularies, whereas topic models require stable corpus-level terms. We introduce LSRTopic, a framework that maps and aggregates contextual sparse activations into the target corpus vocabulary, producing semantically enriched and lexically interpretable document--term matrices for factorization methods such as NMF. Experiments on six benchmark datasets against 10 baselines show that LSRTopic achieves the strongest overall performance across coherence, embedding-based proximity, topic uniqueness, and document-representation separability. Additional LLM-as-judge word-intrusion evaluation with human validation, ablations, and efficiency analysis confirms that resolving vocabulary misalignment enables topic models that jointly support contextual semantics, interpretability, and practical efficiency.