#machine learning
Jul 2026
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
This work investigates the expressiveness of general query/key rotations and finds that any normalized continuous positive-definite attention modulation function can be approximated by random rotations induced by its own Fourier transform, which is term Random Fourier Rotations.
Yi-Wen Chen, Joshua Ainslie, Krzysztof Choromanski et al.
· arXiv.org · 0 citations