Recursion Transformers are studied, reusing a shared block across depth to scale compute, together with factorized embeddings to reduce vocabulary-map parameters, and find that they outperform standard Transformers at 10M and 100M words, while remaining competitive with BabyLM Challenge 2025 winners.
Serdar Gülbahar, Lukas Edman, Alexander Fraser· 0 citations
This work examines recent byte-level models and their failure to perfectly manipulate characters, finding that the hierarchical design itself limits character-level understanding, with pure byte-level models consistently outperforming hierarchical variants on character manipulation tasks.
By thoroughly unifying 36 English hate speech datasets spanning multiple labeling schemes, this work fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate speech mitigation, demonstrating not only state-of-the-art performance on in-domain benchmarks but also substantial improvements in cro...
Lukas Edman, Daryna Dementieva, Alexander Fraser· 0 citations
Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al...
Daryna Dementieva, N. Babakov, Kathy Hammerl et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
