Disentangling the Expressivity of RoPE
It is found that, if every rotary component is periodic, RoPE transformers recognize exactly the languages definable in past temporal logic with modular predicates.
We have 2 of 36 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
It is found that, if every rotary component is periodic, RoPE transformers recognize exactly the languages definable in past temporal logic with modular predicates.
LOTUS is, to the authors' knowledge, the first latent-CoT method to bridge the gap to explicit CoT at the 3B scale, while cutting thought-phase latency by 2.5x-6.9x from compact math expressions to natural language.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.