Preprint
Aug 2026
Disentangling the Expressivity of RoPE
It is found that, if every rotary component is periodic, RoPE transformers recognize exactly the languages definable in past temporal logic with modular predicates.
Selim Jerad, Anej Svete, Jiaoda Li et al.
· 0 citations