Skip to content

DeFA: Dependency-Guided Failure Attribution for LLM Agents

Oct 2026 · 2 citations · ⚡ 1 influential · 28 references
Computer Science

TL;DR

DeFA, a dependency-guided framework for agent failure attribution, achieves the highest responsible-agent and exact step accuracy with all evaluated backbones, and the highest failure-mode accuracy among taxonomy-aligned methods on Pro.

Abstract

Errors in LLM agent executions and their visible consequences can be separated by many steps, making decisive-error localization a matter of understanding both step content and step dependencies. We introduce DeFA, a dependency-guided framework for agent failure attribution. DeFA first combines protocol relations and semantic dependencies into an event dependency graph spanning the trajectory. It then identifies events that may violate task requirements and traces their sources and subsequent effects to construct a failure propagation graph. Finally, DeFA uses step evidence and the steps'roles in failure propagation to identify the decisive error, responsible agent, and error category. To support long trajectories, DeFA partitions executions into segments and combines the current segment's detailed content with summaries of the other segments, giving local diagnosis access to global execution context. Across Who and When and the Who and When Pro text subset, DeFA achieves the highest responsible-agent and exact step accuracy with all evaluated backbones, and the highest failure-mode accuracy among taxonomy-aligned methods on Pro. Further experiments on image and video trajectories demonstrate its applicability to multimodal failure attribution. Ablations support the contributions of segmentation, the event dependency graph, and the failure propagation graph. Using DeFA's diagnostic feedback for skill evolution in Trace2Skill improves downstream task accuracy by 6-15 percentage points over the native pipeline, showing that the diagnoses can also support agent improvement on subsequent tasks.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems

LLM-based multi-agent systems (MASs) are increasingly used to solve complex tasks through coordinated reasoning, tool use, and interaction with external resources. However, attributing failures in such systems remains challenging because the observed outcome often does not directly reveal the error responsible for the...

Jia-Qi Liao, Yuan-Zhao Zhai, Huan-Xi Liu et al. · 0 citations
#machine learning Preprint Oct 2026

Can Failed Trajectories Reveal Where Agents Go Wrong? Agentic Failure Attribution Without Step-Level Labels

Failure attribution for LLM agents aims to identify decisive steps responsible for task failure, which is critical for debugging and improving agentic systems. Existing methods rely on costly LLM prompting, step-level error annotations, or verified successful trajectories that lack failure signals for assistance. In th...

Do Nguyen-Thanh, Jun-Lin Fang, Bing Li et al. · 1 citation

DAAF: From Failure Localization to Editable System Assets in LLM Agents

The Detection-Aware Attribution Framework is proposed, which learns the effects of valid attribute replacements and amortizes this intervention evidence into deployment-time diagnosis, and shows that intervention-grounded attribute attribution can connect failure localization to executable system repair.

Xiao-Yang Yuan, Qi Liu, Yu-Bin Ruan et al. · 2 citations
#artificial intelligence Preprint Oct 2026

Trace-to-Architecture: Capability Bottleneck Diagnosis for Harness Revision

Improving an LLM agent requires knowing what to change when it fails. Similar failures can require different fixes, and resolving one problem can expose another. We introduce Trace-to-Architecture (T2A), a framework that uses execution trajectories to determine what to change in an agent's harness and when those change...

Wen-Feng Wang, Jian-Rui Shi, Min-Gang Guo et al. · 0 citations
#artificial intelligence Preprint Oct 2026

When Messages Go Wrong: Understanding Communication Faults in LLM-Based Multi-Agent Systems

LLM-based multi-agent systems (LLM-MAS) use specialized agents that exchange messages to accomplish complex tasks. Their reliance on networked protocols, remote tools, and model endpoints exposes communication to message loss, delay, and context expiration. Because agents often exchange open-ended natural language, ano...

Jin-Hao Zhou, Xiao-Yue Zhang, Jiongchi Yu et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.