Book Open access

Beyond Language Processing: LLMs Rules-Injected Instruction Tuning for Traffic Prediction

Weihao Jiang Huizhao Wang Zhihui Hu Wen Hu Yao Fu Jiang Zhu Jun Xiao

Aug 2026 · Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · 0 citations · 26 references

TL;DR

Rules-Injected Instruction Tuning (RuleGPT), a paradigm that injects IF–THEN rules extracted from training data directly into the LLM's instruction sequence, allows the LLM to focus on higher-order, context-dependent interactions.

Abstract

Large Language Models (LLMs) demonstrate strong capabilities in contextual integration and multi-step reasoning, which endow them with the potential to model heterogeneous traffic data. However, the knowledge acquired during LLM pre-training is primarily qualitative and broad, and does not provide the fine-grained, context-specific quantitative dependencies required for accurate traffic prediction. This mismatch between pre-training knowledge and task requirements can reduce the effectiveness of instruction tuning, the standard approach for adapting general-purpose LLMs to downstream tasks. To address this issue, we propose Rules-Injected Instruction Tuning (RuleGPT), a paradigm that injects IF–THEN rules extracted from training data directly into the LLM's instruction sequence. For each sample, relevant rules are retrieved and encoded as learnable rule tokens that are concatenated with the input, providing explicit guidance to the model. By encoding low-order dependencies in this way, RuleGPT allows the LLM to focus on higher-order, context-dependent interactions. Experiments on real-world traffic datasets demonstrate that RuleGPT consistently improves prediction performance over standard instruction tuning and strong forecasting baselines. The code is available at https://github.com/tigerbunny2023/RuleGPT.

DOI Semantic Scholar

Read PDF

Similar papers

Preprint Aug 2026

Beyond Factual Knowledge: Benchmarking and Learning Step-Level Procedural Rule Reasoning in Large Language Models

DynaRule is proposed, an end-to-end framework that injects the given rules into the KV cache and turns retrieval into an internal, learnable, step-wise process, and can re-attend to the most relevant rules at each step, dynamically replacing outdated ones to support more stable multi-step reasoning.

Bo-Han Yu, Pengfei Cao, Chen Han et al. · 1 citation

Jul 2026

PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference

Results show that learned token-level handoffs can reduce LLM use while preserving strong reasoning performance, and show that learned token-level handoffs can reduce LLM use while preserving strong reasoning performance.

Niqi Lyu, Pengtao Shi, Wei Qiu et al. · arXiv.org · 0 citations

Conference Open access 2026

A Survey on Context Injection Strategies for Long-Context Language Models: Three Perspectives

This survey argues that context injection strategy, rather than context capacity, is the defining research challenge for long-context LLM deployment, and proposes a three-axis analytical framework revealing that injection performance is jointly governed by selection, representation, and scheduling.

Aicha Dakir, M. El Hajji, Tarek Ait Baha et al. · EPJ Web of Conferences · 0 citations

Review Open access Aug 2026

Large Language Models: From Internal Architecture and Distributed Training Optimisation to Adaptation Strategies

This paper presents a unified technical survey of Large Language Models (LLMs), connecting three layers of the modelling pipeline that existing surveys address in isolation: internal architecture, distributed training optimisation, and downstream adaptation. Its organising principle is the dependency between these laye...

Martin Lukáč, F. Duchoň, Jakub Ivan et al. · Applied Sciences · 1 citation

#machine learning Preprint Aug 2026

OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling

OctoLong is introduced, a context engineering pipeline that instruments an AST parser, a language server backend, and a package manager to facilitate the recursive retrieval of code references, enabling the curation of dependency-rich code contexts of millions of tokens in length.

Indraneil Paul, F. Helm, Goran Glavas et al. · 1 citation

Conference Jul 2026

AdaptiReason: Adaptive Chain-of-Thought Compression for Efficient Large Language Model Inference

Large Language Models (LLMs) trained using Chain-of-Thought (CoT) supervision have achieved state-of-the-art performance on complex reasoning tasks. However, the generation of long reasoning chains introduces substantial computational overhead during inference, limiting their deployment in low-latency and resource-cons...

V. A, Mithaguru, Amrita Kundu et al. · 2026 4th International Confe... · 0 citations