Token compression combines content from several positions, yet rotary position embeddings usually assign the merged token one coordinate. We ask what positional information must survive later merges. Aperture stores Fourier moments of the token's weighted support at the model's rotary frequencies. We prove that these m...
Pre-tokenisation restricts which text fragments can become prediction units, but its compression cost is obscured when tokenisers are compared only under the same boundaries. We measure this cost by bounding the minimum token count from both sides, with and without a regular-expression boundary rule. Nonnegative prices...
It is shown that split invariance forces the local mass factor to be linear for any fixed positive attention kernel, provided that factor is nondecreasing.
Three properties of a world model are distinguished: the distribution it approaches, the rate of approach, and the conditional dynamics it learns, which derive an absolute convergence bound from finite initialization banks and control departure from the reference through conditional action-space divergence.
Yu-Hao Du, Shu-Nian Chen· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.