The first ChineseBabyLM Challenge, organized as part of NLPCC 2026, asked participants to train language models from scratch using no more than 102M Chinese words and provides a benchmark for advancing data-efficient and cognitively plausible approaches to Chinese language modeling.
Abstract
This paper presents the first ChineseBabyLM Challenge, organized as part of NLPCC 2026. The challenge asked participants to train language models from scratch using no more than 102M Chinese words. The models were evaluated on three tracks: natural language understanding, cognitive alignment, and Hanzi knowledge. There were no restrictions on tokenizers, model architectures, or the number of training epochs. Eighteen teams submitted 28 distinct models, generating 74 result files. The overall-winning team used a DeBERTa-v2 architecture and introduced an auxiliary pinyin-prediction objective during pretraining. Several submissions also explored curriculum-learning strategies and architectural innovations. Overall, the challenge provides a benchmark for advancing data-efficient and cognitively plausible approaches to Chinese language modeling.
System One models such as Jev offer an efficient alternative to generative language models for tasks that require decisions rather than open-ended responses. However, existing Jev models exhibit limited Chinese-language decision accuracy, restricting their utility in both general and specialized settings. In this paper...
Ze-Xiao Wang, Zi-Hao Zhang, Xu-Dong Wang et al.· 0 citations
Ufakzeka-1, a 151M-parameter (182M with embeddings) decoder-only Turkish language model pretrained from scratch on 13.5B tokens of openly licensed text and instruction-tuned for chat, is described and three findings are believed to transfer to other small-model efforts.
The experimental results show that even 130M-parameter models benefit from including the MTP task in the pre-training objective, and hold even under severe data constraints, as demonstrated on both zero-shot benchmarks and downstream tasks.
A large language model (LLM) is an AI model (typically a neural network) trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate, and analyze text in many contexts. They are the basis for many modern chatbots, such as Chat...
Seyed Alireza Mousavian Anaraki, Randall Spain, Stefan Hegselmann· 0 citations
Spell correction is still a challenging problem for many languages, especially low-resource languages (LRLs). While pre-trained language models (PLMs) have been employed for spell correction, there has been no proper comparison across PLMs. We present the first empirical study on the effectiveness of the three types of...
Akesh Gunathilake, N. Karunarathna, Tharusha Bandaranayake et al.· Moratuwa Engineering Researc...· 0 citations
The retrieval-augmented many-shot translation pipeline from the AmericasNLP 2026 system is adapted to translate between English and eleven North-Eastern Indian languages in both directions to solve the WMT26 Low-Resource Indic Language Translation shared task.
Aashish Dhawan, Christopher Driggers-Ellis, Dzmitry Kasinets et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
