AdaptiReason: Adaptive Chain-of-Thought Compression for Efficient Large Language Model Inference

PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference

Results show that learned token-level handoffs can reduce LLM use while preserving strong reasoning performance, and show that learned token-level handoffs can reduce LLM use while preserving strong reasoning performance.

Sprint or Delve: A Distribution-Aware Approach to Efficient Reasoning

OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping

Large Language Models (LLMs) have achieved remarkable success in complex reasoning tasks through Chain-of-Thought (CoT) prompting. However, these models often exhibit"computational overthinking,"generating redundant reasoning steps that increase latency and cost without improving accuracy. Recent studies suggest that CoT trajectories can be significantly pruned, yet existing methods often rely on forcing a static thinking budget, heuristic filtering, sub-optimal early exit via classification, or expensive re-training. In this paper, we introduce OS-Pruner, a lightweight plug-in framework that formulates chain-of-thought pruning as an optimal stopping problem. Given a reasoning prefix, OS-Pruner learns whether further reasoning is worth its token cost by optimizing an explicit utility that trades off final-answer accuracy against generated length. Our novel formulation enables the model to dynamically assess the sufficient point of termination for a reasoning chain. OS-Pruner is designed to be lightweight during both training and inference, and to provide users with fine-grained control over the reasoning-effort vs. accuracy trade-off. On diverse reasoning benchmarks and base models, OS-Pruner achieves 20-60\% reduction in generation length with minimal accuracy sacrifice.

Lightweight reasoning models for NER

While large language model based approaches for Named Entity Recognition have advanced through fine-tuning and in-context learning, they often suffer from mediocre performance and a significant lack of interpretability. Although reasoning models employing Chain-of-Thought mechanisms demonstrate superior inferential capabilities, their practical deployment is hindered by high computational overhead and logical redundancy. To address these limitations, this paper proposes Lite- CoNER, a lightweight NER framework that achieves an effective balance between recognition accuracy and inference efficiency. By leveraging knowledge distillation and reasoning simplification, the framework transfers the deep reasoning expertise of teacher models—specifically in entity boundary determination and category logic—to lightweight student models. We first generate high-quality reasoning data through prompt engineering and introduce an LLM-based evaluator to prune logical redundancy and compress reasoning trajectories. Finally, we apply LoRA for parameter-efficient finetuning, enabling the student model to internalize the distilled reasoning capabilities. Experimental results across multiple benchmark datasets demonstrate that Lite-CoNER significantly outperforms traditional fine-tuning methods while maintaining high inference speeds. Furthermore, the simplified reasoning trajectories provide a transparent view of the decision-making process, proving that lightweight models can effectively inherit complex logic through structured distillation.

CLARO: Controlled Attribute-Driven Reasoning Optimization for Efficient Chain-of-Thought

Large language models exhibit strong reasoning capabilities but often require significant computational resources due to verbose, un-structured Chain-of-Thought outputs. Recent approaches guide reasoning length through to-ken penalties or truncation, risking the omission of necessary steps. We posit that conciseness should be an emergent property of structured thought, rather than a result of artificially forced brevity. To this end, we first demonstrate that Attribute-Guided Prompting , a lightweight zero-shot strategy, improves reasoning performance while reducing inference cost. Building on this foundation, we introduce C ontro l led A ttribute-Driven R easoning O ptimization ( CLARO ), a reinforcement learning framework designed to internalize these benefits. CLARO guides models to embed high-quality structural attributes, such as read-ability, math density, syntactic compression, and low redundancy, within a user-defined to-ken budget. The proposed method outperforms state-of-the-art baselines across diverse benchmarks, yielding accuracy gains of up to 63.6%, demonstrating that guiding generated output language structure enhances reasoning. Overall, our findings establish that optimizing the thought process structure refines reasoning efficacy, with computational efficiency emerging as a derivative benefit of a clearer thought process. Code and models are available at https://github.com/odedsc/CLARO .

Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs

Large language models (LLMs) achieve strong generation and reasoning performance, but the Transformer architecture incurs high inference cost. Existing acceleration methods often rely on task-specific fine-tuning or training from scratch, increasing adaptation cost and limiting cross-task usability. We present an Adaptive Depth Sparse Framework (AdaDSF) that converts off-the-shelf pre-trained LLMs into depth-sparse models without full retraining. Our key insight is that layers contribute unequally to representation transformation, characterized by the cosine similarity between layer input and output hidden states. Based on this, AdaDSF assigns layer-wise token retention ratios from similarity statistics, uses a lightweight router to select informative tokens at each layer, and introduces a feature-preserving alignment objective to match intermediate and final representations between sparse and dense models. On GPT-NeoX and Qwen2.5 over language modeling and commonsense reasoning, AdaDSF substantially reduces inference FLOPs while preserving performance close to dense counterparts. Under comparable sparsity, AdaDSF consistently yields smaller accuracy degradation than strong baselines including MoD, D-LLM, and DLO.