Large Language Models (LLMs) often face challenges in performing reliable multi-hop reasoning due to issues such as incomplete evidence chains and hallucinations. Incorporating knowledge graphs (KGs) can mitigate these problems, but existing approaches either suffer from suboptimal accuracy or are computationally expensive. To address these issues, we propose Reasoning Path Retrieval for RAG (RPR-RAG), a novel KG-based retrieval framework that incrementally builds a subgraph from the knowledge graph, extracts explicit reasoning paths, and provides them as structured external evidence to downstream LLMs. The experimental results on WebQuestionsSP (WebQSP) and Complex WebQuestions (CWQ) indicate that RPR-RAG achieves competitive Hit and F1 in multi-hop reasoning tasks, while maintaining runtime, LLM call frequency, and token usage at reasonable levels. Moreover, without additional task-specific training, RPR-RAG also shows strong zero-shot performance on MetaQA. RPR-RAG is built on a lightweight embedding model which can be trained and executed on a single consumer-grade GPU (
e.g
., RTX 3060, 6 GB). Ablation studies reveal that the path validity evaluation and stopping criterion play important roles in retrieval quality and efficiency. RPR-RAG is compatible with a range of backbone LLMs, from smaller 7B models to larger models such as GPT-5, providing a practical and interpretable framework for KG-grounded reasoning tasks. The source code is available at
https://doi.org/10.5281/zenodo.19334059
.
Large Language Models (LLMs) have strong capabilities in mathematical and logical reasoning. However, deploying LLMs is difficult for users with limited hardware. One solution is distilling reasoning skills from LLMs to Small Language Models (SLMs, ≤1B parameters). Still, student models often make five common errors: calculation errors, missing reasoning steps, symbol mistakes, semantic misunderstandings, and coherence errors.This paper proposes Prefix-Description-CoTD (PD-CoTD), an improved distillation method to address these issues. First, structured prompt templates guide LLMs to generate complete reasoning steps. This creates high-quality datasets to help SLMs learn effective reasoning patterns. Second, we introduce a step-by-step optimization loss function. This loss function separates reasoning path modeling from final answer prediction. It helps the model focus on reasoning clearly while optimizing results. Experimental results show that PD-CoTD significantly enhances the mathematical reasoning abilities of SLMs. It achieves state-of-the-art performance and greatly reduces the five common errors. PD-CoTD provides an effective solution for deploying powerful reasoning in resource-constrained environments. It also supports the broader application of lightweight models in academic research and education. Our code implementation and data are available at https://github.com/gameking2159/PD-CoTD .
Zhe Wang, Long Zhang, Fengbo Zheng et al.· International journal of sof...· 0 citations