GRADE: Graph Representation of LLM Agent Dependency and Execution
TL;DR
This work recovers the dependency layer from six tool-use, coding, and web corpora and measures its failure-prediction gain over run size, finding structure can pass for signal it does not carry and structure can pass for signal it does not carry.
Abstract
Traces record what LLM agents did, leaving implicit what each step relied on. GRADE holds both in one graph; six published graph views are its projections. Execution edges come free; each costly dependency edge is graded as observed in the trace, declared by added logging, or inferred under an assumption, a schema we test one grade at a time. We recover the dependency layer from six tool-use, coding, and web corpora and measure its failure-prediction gain over run size. First, over a linear run-size baseline, it improves prediction on three corpora, but the gain depends on the learner and task sampling. On held-out corpora the layer stays above chance where run size does not, under one learner, with the same task-sampling caveat. Second, structure can pass for signal it does not carry: a layer inferred under full history restates run size. Within a corpus, a preregistered test does not separate the recovered layer from a degree-matched counterfeit that rewires each step's dependencies but keeps their number. The recovered layer's largest transfer lead over it reverses when the held-out corpus's task family is withheld. Before crediting structure, grade each edge, keep a run-size baseline, and score degree-matched attachment controls.
