Jun 2026· arXiv.org· Vol abs/2606.25568· 0 citations· 25 references
Computer Science
TL;DR
Riazi-8B, an Urdu mathematical reasoning model developed through a two-step adaptation process comprising continued pre-training on Urdu Wikipedia and supervised fine-tuning on Urdu Chain-of-Thought data derived from GSM8K, demonstrates that combining Urdu language adaptation with reasoning-focused fine-tuning is an effective strategy for extending mathematical reasoning capabilities to low-resource languages.
Abstract
Recent LLMs demonstrate strong mathematical reasoning capabilities, but existing gains rely heavily on English-centric training resources and benchmarks. As a result, reasoning performance degrades substantially in low-resource languages such as Urdu, where reasoning-oriented datasets and adapted models remain scarce. Urdu lacks both reasoning-oriented resources and models adapted for multi-step mathematical problem solving, limiting the applicability of recent progress to Urdu-speaking users. We address this gap through Riazi-8B, an Urdu mathematical reasoning model developed through a two-step adaptation process comprising continued pre-training on Urdu Wikipedia and supervised fine-tuning on Urdu Chain-of-Thought data derived from GSM8K. We evaluate Riazi-8B on MGSM-Urdu against existing Urdu instruction-tuned models. Our results show consistent improvements in answer correctness, reasoning quality, response completeness, and Urdu generation. Our findings demonstrate that combining Urdu language adaptation with reasoning-focused fine-tuning is an effective strategy for extending mathematical reasoning capabilities to low-resource languages.
Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al., 2025) dataset represents a significant step forward, yet its coverage is still limited to 18 only high-resource languages. To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional {underrepresented languages spanning 6 language families -- ranging from mid-resource to extreme low-resource settings. We constructed the dataset through a human-curated pipeline, where native speakers thoroughly validated pre-computed translations. Using PluraMath, we then benchmark 27 reasoning LLMs across four model scales -- small, mid-size, large, and closed-source ensembles -- probing the multilingual mathematical reasoning capabilities of state-of-the-art models under diverse linguistic conditions. Our fine-grained analysis confirms a persistent gap in mathematical reasoning performance between high-resource and underrepresented languages, with stronger results largely associated with better instruction-following ability. We fully open-source our dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to multilingual benchmark development for underrepresented communities.
Daryna Dementieva, N. Babakov, Kathy Hammerl et al.· 0 citations
Large Language Models (LLMs) have strong capabilities in mathematical and logical reasoning. However, deploying LLMs is difficult for users with limited hardware. One solution is distilling reasoning skills from LLMs to Small Language Models (SLMs, ≤1B parameters). Still, student models often make five common errors: calculation errors, missing reasoning steps, symbol mistakes, semantic misunderstandings, and coherence errors.This paper proposes Prefix-Description-CoTD (PD-CoTD), an improved distillation method to address these issues. First, structured prompt templates guide LLMs to generate complete reasoning steps. This creates high-quality datasets to help SLMs learn effective reasoning patterns. Second, we introduce a step-by-step optimization loss function. This loss function separates reasoning path modeling from final answer prediction. It helps the model focus on reasoning clearly while optimizing results. Experimental results show that PD-CoTD significantly enhances the mathematical reasoning abilities of SLMs. It achieves state-of-the-art performance and greatly reduces the five common errors. PD-CoTD provides an effective solution for deploying powerful reasoning in resource-constrained environments. It also supports the broader application of lightweight models in academic research and education. Our code implementation and data are available at https://github.com/gameking2159/PD-CoTD .
Zhe Wang, Long Zhang, Fengbo Zheng et al.· International journal of sof...· 0 citations
Reasoning Language Models (RLMs) achieve their strongest performance when they reason in English, the language for which reasoning-oriented training data is most abundant. However, reasoning trace is a clue for model interpretability and safety, and useful in practice for both the model users and for model developers. Thus, it is desirable to be able to develop a model that reasons in a language of the user's choice, while still maintaining strong reasoning performance. To this end, we study the feasibility of training a model that reasons in Japanese. We develop a Japanese-reasoning variant of Qwen-3-Swallow-8B, which is a Japanese LLM continually pretrained from Qwen-3-8B, with GRPO and evaluate it across coding, math, and science benchmarks. The study shows that reasoning-language control is feasible by training a Japanese continually pretrained model with GRPO. However, its performance is at best on par with strong English-reasoning baselines on several benchmarks. We also evaluate the trained model on Japanese cultural benchmarks and observe that the model's performance is worse than the baseline models, suggesting that the reasoning in Japanese does not immediately improve performance on culturally relevant tasks for free.
Large Language Models (LLMs) often exhibit limited performance on mathematical reasoning tasks. This paper proposes an activation steering (AS) method based on single-vector ablation to enhance mathematical reasoning by injecting a carefully constructed steering vector into the model’s residual stream. Specifically, the AS direction is constructed from the activation difference between mathematical and general-domain samples and is injected into designated transformer layers during inference. Experiments on Llama-3-8B-Instruct demonstrate that the proposed method improves mathematical reasoning accuracy from 11.0% to 39.6%, while simultaneously enhancing general capabilities as well. However, transfer experiments on Qwen2.5-7B-Instruct fail to achieve comparable improvements, revealing the strong model dependency of the proposed approach. Furthermore, the effects of layer selection, steering coefficient, positional window, and steering vector construction are systematically investigated. This study provides both empirical evidence and theoretical insights regarding the application of activation steering methods.
Yu Han, Panpan Zhang, Bo Zhang· Advances in Engineering Tech...· 0 citations
Addressing the bottlenecks of existing Chain-of-Thought (CoT) methods in complex reasoning of Large Language Models (LLMs), such as reasoning skips, frequent logical illusions, and unbalanced computational costs, this paper proposes an original Adaptive Bidirectional Verification Chain-of-Thought (ABV-CoT) algorithm. This algorithm comprises three core modules: a task complexity-aware forward adaptive reasoning step size generation module, which dynamically matches reasoning granularity with task difficulty; a reverse logical closed-loop verification and error backtracking correction module, which can locate and correct intermediate reasoning errors, reducing logical illusions and error accumulation; and a lightweight gated attention optimization unit, which can improve reasoning performance while controlling computational costs. Comparative and ablation experiments are conducted on four standard complex reasoning datasets, including GSM8K and MATH, using Llama-2-7B, Llama-2-13B, and Qwen-14B as base models. The results show that the ABV-CoT algorithm improves the average inference accuracy by 5.12-19.78 percentage points compared to the mainstream CoT method, reduces inference latency by 36.84% compared to the Tree-of-Thoughts (ToT) method, and reduces memory usage by 26.09%. It exhibits excellent generalization ability on small-to-medium-scale models, providing a new technical path and theoretical support for enhancing the complex reasoning capabilities of LLMs, and has significant academic and engineering value.
Yang Guo, Qian-cheng Guo· 2026 IEEE 2nd International...· 0 citations
Large language models (LLMs) for reasoning generation rely on their own already acquired knowledge. However, knowledge in real tasks is updated in real time, and frequent fine-tuning can be cumbersome. Recent years have witnessed the success of large-scale knowledge graphs, which could serve as an ideal domain knowledge resource. However, these large-scale knowledge graphs cannot be directly applied to LLM reasoning tasks. Moreover, they often struggle to combine the reasoning capabilities of LLM with large-scale knowledge graphs. Existing methods can hardly make use of them because the black-box structure of LLMs, which is difficult to handle. To address these problems, we present a method combining feedback enhancement with knowledge graphs in LLMs, namely FKGLM, which can automatically mining logical rules from knowledge graphs to create a domain knowledge base (KB). Meanwhile, the KB is used to perform minimum inconsistency reasoning on the initial results generated by the LLM, correcting errors in the results and updating the prompt to achieve efficient knowledge augmentation of the LLM. Experiments on three different domain tasks show that FKGLM can effectively integrate LLMs and large-scale knowledge graphs, leading to a significant enhancement in the reasoning capabilities of LLMs.
Yulin Zhou, Yongbin Qin, Chuan Lin· Journal of King Saud Univers...· 0 citations