Jun-Hao Wu

A Review of Long-Context Processing Ability in Large Language Models: Technical Progress and Challenges

It is argued that the ability of long context should not only come from increasing the context window, but also from the ability of the model to locate, integrate and reason about important information in long text.