This work proposes a framework that ensembles the reasoning structure, not just the answers, of multiple LLMs by weighted merging of Directed Acyclic Graphs (DAGs) extracted from reasoning chains by weighted merging of Directed Acyclic Graphs (DAGs) extracted from reasoning chains.
Abstract
Large Language Models (LLMs) explore problems through chain-of-thought, but this exploration is buried in unstructured prose. On high-stakes tasks, users cannot tell which steps are well-supported, which alternatives were seriously considered, or how the final conclusion compares to those the model discarded. We propose a framework that ensembles the reasoning structure, not just the answers, of multiple LLMs by weighted merging of Directed Acyclic Graphs (DAGs) extracted from reasoning chains. We weight each step by how many traces independently attest to it, to return"Consensus Reasoning". Across six benchmarks spanning statutory interpretation, graduate-level science, narrative multi-hop reasoning, and first-order logic, our ensemble outperforms a matched-budget majority-vote baseline, with a maximum accuracy gain of 3.1% on MuSR-MM (narrative multi-hop reasoning). On a single model, the framework matches or exceeds self-consistency at the same trace budget while additionally exposing an inspectable consensus reasoning graph. Ensemble weights correlate with LLM-judge rankings of reasoning quality at Spearman $\rho = 0.30$-$0.51$, and consensus subgraphs are preferred over alternatives leading to the majority-vote answer in 54.4-65.4% of head-to-head comparisons across five of six datasets. We observe that our framework can also be used to analyze diverse reasoning perspectives for a problem.
This work proposes Long Chain-of-Thought Graph Verifier (LCoT-GV), a graph-based framework that represents LCoTs as reasoning graphs, each node in the graph represents a reasoning step and the edges encode semantic and logical relations.
Inference-time search with large language models (LLMs) often concentrates on a small set of structurally or semantically similar trajectories, leaving alternatives underexplored---a failure mode we call \textit{reasoning basin collapse}. We introduce BASIN, a training-free, structure-aware selection method that groups reasoning states into basins and penalizes repeated visits to the same strategy, thereby reallocating search across genuinely distinct reasoning paths under a fixed compute budget. Under matched inference budgets, BASIN improves over Tree of Thoughts (ToT) by up to $+22$pp on Game of 24 and $+6.7$pp on MuSR. A quality-aware variant, QA-BASIN, further improves robustness by preserving high-quality basins when unconditional diversification over-explores. To explain when basin-aware selection helps, we introduce the redundancy gap $\Delta$, which measures how differently search concentrates for correct versus incorrect predictions: standard ToT often operates near $\Delta \approx 0$, while BASIN consistently shifts $\Delta$ positive. More broadly, BASIN suggests structure-aware selection as a simple and general approach to improving inference-time reasoning. Code can be found at https://github.com/GitHubLuCheng/basin.
Despite the success of Large Language Models (LLMs) on reasoning benchmarks, it remains unclear whether their performance stems from genuine logical deduction or the memorization of training patterns. Existing benchmarks often fail to disentangle reasoning from prior knowledge, as tasks grounded in real-world facts allow models to take ''knowledge shortcuts''. In this paper, we propose a novel diagnostic benchmark to decouple knowledge memorization from logical reasoning. Built on the DBpedia KG, our framework constructs multi-hop reasoning chains (from Q1 to Q5) across three task dimensions: Factural Questions (FQ), Counterfactual Questions (CQ) with logically consistent but counterfactual conclusions, and Questions with Similar-Entity Options (SO) to evaluate the dependence on prior knowledge. Questions without Context serve only as an intermediate form: they contain solely queries with no triples or options, so LLMs cannot answer them directly. Our core hypothesis is that genuine reasoning is demonstrated only when a model follows logical rules despite conflicting prior knowledge. Evaluating seven state-of-the-art LLMs (8B to ultra-large) reveals strong prior knowledge dependence, with performance degrading sharply on counterfactual tasks as reasoning depth grows. This work delineates LLM reasoning boundaries and presents a new paradigm for fine-grained capability assessment.
Fangfei Yan, Jianbo Yao, Michael K. Chen et al.· Proceedings of the 32nd ACM...· 1 citation
TKFQA, a factuality consistency benchmark comprising 10,130 question-answering (QA) pairs grounded in tables, texts, and knowledge graphs, is introduced and ORLF, an LLM-agnostic training framework that models cross-context topological relations through knowledge-specific latent vectors is proposed.
Shibo Chu, Yuze Liu, Tiehua Zhang et al.· 0 citations
Constraint-Anchored Reasoning Traces (CART) is proposed, a neuro-symbolic framework that trains MLLMs to interleave natural language reasoning steps with symbolic constraint assertions: lightweight, machine-checkable statements about visual content.
SodaMem is presented, an evidence-grounded temporal graph memory that extracts typed FactEvents with mandatory provenance spans, persists mention time, occurrence time, and validity with SUPERSEDES/CONTRADICTS/UPDATES edges under hybrid lexical-dense indexing and answers via a planner-reader loop that gathers citable evidence before composing a final response.
Fengrong Wan, Chengcan Wu, Ning Lyu· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
