A hypergraph-based paired failure attribution (HPFA) framework that attributes the failure root cause by comparing the hyperedges of the targeted failure reasoning path against a reference successful path is proposed, and the trained attributor consistently improves reasoning accuracy at test time.
Abstract
Reflection is a powerful mechanism for LLM reasoning, yet its effectiveness hinges on accurately attributing failures to specific reasoning steps, a capability that current models notably lack. Existing failure attribution methods either require expensive step-by-step counterfactual testing that scales poorly with trajectory length, or treat reasoning traces as flat sequences that ignore the inherent non-linear logical dependencies. We propose a hypergraph-based paired failure attribution (HPFA) framework that attributes the failure root cause by comparing the hyperedges of the targeted failure reasoning path against a reference successful path. By reducing the search space, our method efficiently localizes root causes and enables scalable synthesis of attribution data for training a lightweight attributor model via supervised fine-tuning and reinforcement learning. Experiments on mathematical reasoning and agentic coding tasks demonstrate that HPFA can dramatically increase attribution accuracy and efficiency, and the trained attributor consistently improves reasoning accuracy at test time, outperforming baselines that lack graph structure or paired analysis.
AFANet is introduced, a lightweight graph-based framework that models interaction trajectories through step-level semantic signals and agent-level relationships and suggests that effective agent failure attribution does not require heavy LLM reasoning and a lightweight, structured approach can achieve strong performance.
Ting-Wei Li, Yuanchen Bei, Xiao Lin et al.· 1 citation
Large language model (LLM)-based multi-agent systems have experienced rapid growth in recent years. Despite their promise, such systems remain fragile, frequently exhibiting reasoning and coordination errors that can lead to system-level failures. Failure attribution in such systems relies on tracing natural language interactions among agents to identify the decisive error, which refers to the earliest action whose correction can reverse system failure. There are two key challenges: 1) Shallow attribution: Existing methods often capture only minor deviations, such as incomplete retrievals or formatting errors, which verification mechanisms can correct, while missing the decisive cause of system failure. 2) Contextual degradation: As the length of the system traces increases, the model's reasoning ability rapidly deteriorates. To address these challenges, we propose DCFA, a training-free framework for failure attribution. DCFA integrates a global module that constructs structured causal-inspired dependency graphs from system traces to identify the initial decisive error, and a local module that applies local counterfactual-inspired reasoning to refine causal-inspired attribution. Experiments on the Who&When benchmark across six LLMs show that DCFA improves step-level accuracy by up to 8.27% over state-of-the-art baselines.
Ze-Hao Wang, Lanjun Wang, Shi-Long Jin et al.· 0 citations
While large language models (LLMs) have achieved impressive gains in commonsense reasoning, they often fall into “associative shortcuts,” failing to distinguish correct answers from plausible but constraint-violating hard negatives. This reliance on semantic priors rather than specific situational constraints limits their fine-grained reasoning capabilities. To address this issue, we propose EcoReason (Evolved Commonsense Reasoning), a graph-guided evolutionary and negative-aware reinforcement learning (RL) framework. First, we introduce Graph-Guided Data Evolution, an iterative data generation strategy coupled with the student model’s training progress. In each round, we use knowledge graphs (KGs) to identify deceptive sibling concepts and employ a teacher LLM to create constraint-heavy question answering (QA) data targeting the student’s current blind spots. As training progresses, the synthesized curriculum becomes increasingly challenging. Second, we propose Negative-Aware Policy Optimization (NAPO), an RL algorithm built upon Group Relative Policy Optimization (GRPO). NAPO identifies “stubborn negatives,” defined as incorrect options that are repeatedly selected by sampled policies within the same rollout group, and applies stronger penalties to these recurring distractor-specific errors. Experiments show that EcoReason substantially improves LLM commonsense reasoning in complex constraint-heavy scenarios.
Xin Guan, Jiuxin Cao, Biwei Cao et al.· IEEE Transactions on Audio,...· 0 citations
Large language model (LLM)-based agents have shown strong potential in solving complex tasks through multi-step reasoning, yet they remain vulnerable to execution failures. Accurate failure attribution is therefore critical for improving agent reliability. Existing topology- and spectrum-based methods exploit trajectory structures but often overlook fine-grained semantics, while LLM-based attribution methods capture semantic cues but suffer from long-context degradation over lengthy trajectories. To address these challenges, we propose DUOTRACE, a plug-and-play detection filter for LLM-based failure attribution. DUOTRACE follows a detect-before-attribute paradigm: it first detects anomalous executions and then supplies focused trajectory evidence to downstream LLM-based attribution methods. For effective VAE-based anomaly detection on agent trajectories, DUOTRACE integrates dual-view semantic-structural node representations, a Tree-LSTM-based trajectory encoder, and prefix-chain- and LLM-based data augmentation to handle heterogeneous nodes, hierarchical execution structures, and limited failure data. Experiments with six LLM-based attribution baselines show that DUOTRACE improves agent-level and step-level attribution accuracy by 8.7% and 7.0%, respectively.
Jia-Yi Zhang, Zexin Wang, DecisionMakingRon Sun et al.· 0 citations
PoTRE (Poly-Topological Reasoning Ensembles), a heterogeneous framework that decouples inference into four agents that achieves improved reasoning performance using similar or fewer inference tokens compared to heavily scaled homogeneous baselines is introduced.
Adaptive Influence Graphs is introduced, a two-stage agentic framework that first transforms a failed trace into a structured graph and then navigates it to identify the critical error and establishes a new state of the art on Who&When, the standard benchmark for multi-agent failure attribution.
Yarden Bakish, Amir Dudai, Roy Ganz et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
