On-Demand Attention: Language Models Know When to Recall
These findings show that pretrained decoding states can support both token prediction and decisions about accessing distant information, allowing models to allocate global computation as needed during decoding, allowing models to allocate global computation as needed during decoding.
Hai-Bo Feng, Ruiqi Liang, Dong-Yang Jin et al.
· 0 citations