It is argued that governing such agents is a runtime problem -- not a model-alignment problem and not a build-time problem -- and five primitives are derived from the questions that must be answered before an action takes effect and after it has: discovery, identity, governance, attestation, and supply chain.
Abstract
Enterprise deployments of autonomous AI agents inherit a control model built for human users and long-lived services, and the fit fails in three specific ways: agent principals are ephemeral, appearing and vanishing faster than provisioning; their actions are selected by a model rather than programmed, so the set of things they may attempt is not known in advance; and the population is discovered rather than provisioned, because anyone who can call an API can create one. We argue that governing such agents is a runtime problem -- not a model-alignment problem and not a build-time problem -- and we derive five primitives from the questions that must be answered before an action takes effect and after it has: discovery, identity, governance, attestation, and supply chain. For each we state what fails if it is absent and why the others cannot structurally supply it. We describe an implementation in which an agent's action is mediated against policy before it takes effect, authorised against a per-tenant action vocabulary, and recorded in a hash-linked signed ledger a third party can verify with the vendor out of the loop. We report what the architecture costs: the enforcement point sits on the request's critical path, identity requires a sidecar per workload, and fail-closed mediation converts availability incidents into denial. We are explicit about implementation status: four primitives are built and running in private pilots, and the fifth is built as separate tooling and not yet integrated into the request path. We keep it in the set deliberately: a five-part decomposition that exactly matches what its authors happened to build is not a taxonomy but a description of a codebase.
Deploying agentic AI in regulated contexts requires knowing two things about a deployment: what the system can do—its agency—and how much it acts without human involvement— its autonomy. Though often treated independently, the two are coupled: at higher autonomy, human error correction is less available, so reliable operation requires constraining agency accordingly, and compliance rules reinforce this by mandating human involvement as the consequences of actions grow. Yet no established approach addresses them jointly as a design problem, leaving practitioners without a principled basis for deciding where oversight should sit and how errors can be caught before they propagate. We introduce a two-dimensional design space in which both dimensions are organised into five operational levels, making the coupling explicit and navigable, and we propose six architectural tactics—checkpoints, escalation, multi-agent delegation, tool provisioning, tool fencing, and write staging—for adjusting a deployment’s position within it. We ground the tactics in a public-sector document classification system, tracing a path from manual operation to near-full autonomy under realistic compliance constraints. Together they offer a shared vocabulary for compliance-aware agentic AI design in which responsibility, auditability, and reversibility are explicit design choices rather than retrofitted properties.
This is the first paper from POLIS, an ongoing research programme studying algorithmic institutions for multi-agent systems, and asks which parts of an AI institution produce safety and how they do it.
This paper proposes two complementary artifacts: an Agency Justification Record (AJR) helps teams decide when an agent is warranted over simpler alternatives and an Agentic Delegation Policy (ADP) captures what must be specified for safe and effective development.
Chetan Arora, Andreas Vogelsang, Abbishek Sharma· arXiv.org· 0 citations
Autonomous AI agents now hold execution authority over high-consequence enterprise actions in finance, healthcare, and infrastructure, where catastrophic failures are rare yet dominate systemic risk and probabilistic content filtering does not constitute a reference monitor over what is executed. This paper introduces L-DREA, a deterministic runtime-enforcement architecture that generalizes Anderson’s 1972 reference-monitor primitive from mediation of data access to mediation of externally effective action. L-DREA separates capability generation from execution authority, binds every candidate action to an epoch-keyed Permit-to-Act token, and interlocks externalization through a commit-before-actuate substrate. Five structural properties generalize Anderson’s primitive — complete mediation, tamper-resistance, verifiability, non-compensatory aggregation, and epistemic bounding — and six runtime invariants are established analytically, the first additionally mechanized in TLA+ with a released TLC log. A software (Tier-S) reference implementation is evaluated on two disjoint evidence tracks: a seeded synthetic corpus of 1,217,906 runtime proposals (360,000 adversarial), and the public ULB credit-card dataset (284,807 transactions) as a golden-oracle authorization trace, plus a blind committed-before-label-reveal protocol on three public datasets. Across both tracks, a 120,000-attempt full-knowledge adaptive attacker, 2,394 injected runtime attacks, live revocation and watchdog suites, and an executed offline AgentDojo run, zero unauthorized externalizations were observed; exact one-sided Clopper–Pearson and Wilson upper bounds accompany every zero-event claim, with replay determinism of 100.0000% over 1,217,906 cycles and 24,912 Ed25519-signed permit tokens runtime-verified at 100% integrity. All results are bounded to the documented threat surface and released as a seeded, one-command reproducible artifact; hardware substrates and HSM key custody are specified, not claimed.
Charting these challenges provides a roadmap toward trustworthy autonomous agent deployment: security must become a verifiable property of the architectures, protocols, and runtimes that govern agent behavior, rather than an optional layer of guidance.
AI Agents are increasingly deployed in real-world settings, where they interact with external tools and make sequential decisions with limited human oversight. This creates a pressing need for reliable and auditable explanations of what an agent did and why. However, traditional Explainable AI (XAI) methods fall short of providing the process-level transparency required for such interactive, multi-step systems, motivating a paradigm shift toward approaches specifically designed for AI Agents. To address this gap, we present a post-hoc XAI framework that transforms a lengthy agent's execution trace into a structured report and a faithful natural-language explanation explicitly grounded in its observable behavior. Because it relies solely on execution traces, the framework applies across different agent architectures, environments, and tasks. Human and automated evaluations across multiple benchmarks and architectures show that our framework produces high-quality, trace-faithful explanations while reliably identifying unsupported claims, unjustified actions, and evidence gaps, outperforming naive LLM-generated explanations.
Vittoria Vineis, Fabiano Veglianti, Lorenzo Antonelli et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
