Daniel Jurafsky

Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models

This work explores a self-supervised framework that encourages models to predict concepts, approximated as sets of semantically equivalent tokens, suggesting that concepts enhance semantic alignment while preserving language modeling quality.