Context Language Models
We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to maintain in context, and naturally extends t...
Ru-Lin Shao, Shannon Zejiang Shen, J. Yin et al.
· 0 citations