Closed-World Trustworthy Copilots for Legal and Regulatory Text

Abstract

Applying large language models (LLMs) to regulated domains such as financial and legal compliance is fundamentally a software engineering problem: an answer produced from a model’s parameters is, at the surface, indistinguishable from one grounded in the authoritative source, yet only the latter can be audited. Prior studies report legal-fact hallucination rates of 58–88% for general-purpose models and 17–33that trustworthiness here should be engineered into the surrounding system rather than sought from a stronger model, through confinement, which restricts the model to a single sealed corpus, and measurement, which quantifies per-claim groundedness. We present the closed-world copilot, a pipeline that seals one regulatory document into a structure-aware index, enforces a citation-and-verbatim-quote contract on every claim, and runs an independent leakage verifier combining a deterministic quote check, an isolated entailment judge, and a parametric-leakage probe. On the EU Markets in Crypto-Assets (MiCA) regulation, confinement improves groundedness and abstention over a baseline LLM and vanilla retrieval-augmented generation (RAG), and a synthetic-amendment test shows that an unconfined model reports memorized and potentially stale values. A second study on the Contract Understanding Atticus Dataset, reproduces this result and quantifies its cost in coverage. We position the approach as a trust layer above retrieval rather than a replacement for it.