LLM-maintained wiki systems accumulate structured knowledge by having a language model incrementally build and revise a persistent wiki layer over raw evidence sources. When such systems support decision-making in regulated domains, each output must be traceable to grounded evidence through an auditable chain. This paper presents a provenance-native architecture in which a dedicated provenance layer is embedded in the wiki’s operational loop, recording evidence bindings, claim versions, rule compilations, and decision traces as side effects of normal system operations. The architecture is organized around four technical components: (1) a typed provenance graph that binds claims to evidence passages with confidence-weighted edges; (2) a dual-layer diff mechanism combining textual and semantic comparison to detect silent LLM edits; (3) three confidence propagation strategies over multi-hop evidence–claim–rule–decision chains, compared on their suitability for regulatory communication; and (4) a retraction propagation engine that identifies all downstream dependents when upstream evidence is invalidated. We evaluate the system on a 112-document corpus producing 3174 graph nodes and 4398 edges. In controlled retraction experiments, the engine achieves 100% downstream recall with zero false propagation in under 600 ms per event. A sensitivity analysis over the human-review threshold parameter γ reveals a sharp phase transition in the fraction of flagged decisions, providing a concrete basis for negotiating review policies with regulators. An automated gap analysis shows the architecture demonstrates alignment with five of eight requirements derived from the EU AI Act and FDA AI/ML guidance, and we report module-level evaluations of claim extraction, evidence binding, and silent-edit detection together with a measured baseline comparison on retraction propagation. Code and experimental protocols will be released publicly upon publication.
: High-stakes clinical decision support (CDS) demands a property that aggregate accuracy cannot capture: a trace that a clinician who was not in the room can inspect layer by layer when the system is wrong. We argue that the way to obtain this property is to refuse to entangle the large language model (LLM) with the rest of the pipeline. We propose KRD (Knowledge–Rule–Decision) , a four-component architecture that separates fact extraction, a compile-time clinical knowledge layer in the spirit of the LLM Wiki pattern of Karpathy, a rule layer of hand-written contraindications and heuristics, and a decision interface whose compose method short-circuits to a rule-cited blocking response whenever any hard violation fires. We evaluate KRD against a pure language model, a retrieval-augmented language model, a rule-only system, and a light hybrid on a benchmark of 32 type-1 diabetes scenarios. A strict version of the unsafe-suggestion rate stratifies the five systems monotonically into four distinct tiers from 0.867 down to zero, with S4 and S5 tied at the floor; the full KRD stack and the light hybrid reach the hard-safety ceiling together; KRD leads the light hybrid on evidence trace completeness by 25% relative and on reviewer correction burden by 12% relative, both directionally clear and borderline significant under bootstrap intervals; and KRD issues 17 language model calls per benchmark pass against the light hybrid’s 32, a 47% reduction that is a direct consequence of the architectural choice to evaluate the rule layer before invoking the model. We also report honestly that the evidence gate is inert on this benchmark because every compiled concept is graded A or B, and we trace five fact-extraction failures to a single field and a single linguistic pattern. The contribution is not that KRD is universally optimal but that layer-wise auditability is a design discipline whose cost in this setting was lower than its critics would have predicted.
Bailing Zhang, Genlang Chen· Journal of Intelligent Medic...· 0 citations
. Large Language Models (LLMs) are increasingly used to maintain persistent, domain-specific knowledge bases—a paradigm in which assertions must remain accurate as the field evolves. Existing staleness detection treats each assertion independently, missing a structural reality: when a foundational concept becomes outdated, every dependent concept inherits some degree of that staleness through ontology relationships. We present WikiMonitor-Onto, a lightweight propagation layer built on WikiMonitor that models staleness as a signal flowing through a domain ontology graph. We extract a concept graph of 642 nodes and 487 edges from 61 AI lecture documents, define three typed propagation relations ( is-a , depends-on , related-to ), and propagate staleness via weighted Breadth-First Search (BFS) with exponential hop decay. On a human-annotated gold standard of 62 concepts (25 indirect-stale, 37 fresh), the independent baseline detects zero indirect-stale concepts by construction, while WikiMonitor-Onto achieves precision 0.824 and recall 0.560 at the optimally tuned configuration ( λ = 0.30). Grid search reveals that is-a and depends-on carry equal optimal propagation weight (both 0.90). A sensitivity analysis confirms that propagation is robust to the choice of seed-value distribution, with F1 varying by only 0.10 (0.571–0.667) across four tested distributions. Propagation saturates at hop depth 1 under conservative thresholds, suggesting that one-hop propagation suffices for high-precision deployment.
Bailing Zhang· Journal of Advances in Artif...· 0 citations
A previously unrecognised failure mode in LLM-to-logic compilation is characterise a previously unrecognised failure mode in LLM-to-logic compilation and a practical engineering toolchain for producing verifiable knowledge-base programs from natural-language specifications is demonstrated.
Bai-Ling Zhang· AI Engineering· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
