This work presents an evidence-carrying validation interface: every selected node-shape check returns either a satisfaction trace or failure witness, and shows how programs combine passing and failing evidence to diagnose missing information and guide repair.
Abstract
Programs that consume a knowledge graph they do not maintain, such as applications, authoring platforms, and LLM agents, need to know whether the graph contains the information their task requires. Validating the graph against a schema can answer this question, but existing validation interfaces usually return a conformance bit or failure-oriented report without identifying why checks pass or the partial matches behind failures. We present an evidence-carrying validation interface: every selected node-shape check returns either a satisfaction trace or failure witness. These are mutually recursive objects that retain constraints, cardinality decisions, paths, and supporting triples. We implement this interface in Shifty, an experimental SHACL validator. Against two real-world shape graph corpora, materializing all-pair evidence costs a median 1.54-2.07X conformance-only validation. A case study then shows how programs combine passing and failing evidence to diagnose missing information and guide repair.
A standard verification-based knowledge graph question answering (KGQA) pipeline retains a reasoning path only when retrieved evidence supports it. Requiring evidence makes large language model (LLM) output easier to audit and reduces unsupported answers. The verifier’s two outcomes are not equally informative: acceptance provides strong evidence, whereas rejection on an incomplete graph may indicate either an incorrect path or a missing proof. A one-pass pipeline discards both cases and loses the information contained in the failure. We introduce GRACE (Graph Reasoning with Adaptive Controller and Evidence), which feeds rejection back into retrieval and restores accept–reject symmetry in the search. GRACE assigns a failure type to each rejected path. Its reflective feedback loop (RFL) revises subsequent retrieval, whereas hypothetical bridging (HB) admits budgeted and penalized virtual edges for plausible missing links. A virtual edge remains marked as a hypothesis rather than a verified triple; the bridge budget bounds answer-level hallucination exposure, and the marking makes that exposure observable. With a fixed open-source LLM backbone, GRACE improves KG-grounded reasoning on WebQSP and ComplexWebQuestions. Diagnostic results indicate that RFL and HB recover different types of failure.
SEGRA is introduced, an experience-guided agent for enterprise text-to-Gremlin question answering that integrates intent routing, schema- and taxonomy-grounded query generation, multi-shot decomposition, execution-aware verification, and a curriculum-bootstrapped skill library that reuses verified query patterns.
Saiyue Lyu, Mariam Dundua, Vishaal Kapoor et al.· 0 citations
This work presents an agentic text-to-SPARQL system that goes one step beyond static tool-using agents: a researcher agent that, after each round of inference on a validation set, proposes and tests changes to its own prompts, rules, and tool-orchestration code.
EntailLLM is presented, which validates each LLM-proposed analyst path by entailment: the path is a traversal of the binary's function call graph, the domain knowledge is represented in a separate graph, and verification aligns the two under temporal annotated logic.
Kaustuv Mukherji, J. Patil, C. Payne et al.· 0 citations
ClosureBench is introduced, a constructive benchmark for compositional graph-relational reasoning with programmatically verified ground truth with programmatically verified ground truth: each task's reference answer is computed by executing a program in the Ein tensor-logic language, ensuring machine-verified correctness.
S. Goria· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
