Skip to content
Open access

Risk for AI Agents

Sep 2026 · European Journal of Risk Regulation · 0 citations

TL;DR

Two paths for conceptualising agentic risk are developed: a conservative path, which retains the standard definition of risk and reconstructs it formally for narrow agents; and a governance path, which draws an analogy to the normative systems used to manage biological agents, arguing that sufficiently general AI agents require something closer to law-following than to product-safety assessment.

Abstract

This paper investigates whether the concept of risk applies to AI agents. The EU AI Act, in force since 2024, grounds its regulatory architecture in a product-based model of risk: one that presupposes an intended purpose, (a bounded space of) reasonably foreseeable misuses, and a tractable distribution of potential harms. This paper argues that these three presuppositions fail structurally for sufficiently general agentic systems. Agents whose behavioural space is open-ended by design cannot be assigned a purpose specific enough to anchor risk identification; their misuse space is generatively open, including novel sequences discoverable by the agent itself; and the probability and severity of harm are subject to modification through the agent’s own learning and action on the environment. What fails, however, is the product-style operationalisation of risk (per system, ex ante , anchored in intended purpose), rather than risk as such. Given this diagnosis, the paper develops two paths for conceptualising agentic risk: a conservative path, which retains the standard definition of risk and reconstructs it formally for narrow agents (either as expected deviation from an ideal trajectory, or as the probability-weighted sum of constraint violations); and a governance path, which draws an analogy to the normative systems we use to manage biological agents, arguing that sufficiently general AI agents require something closer to law-following than to product-safety assessment. The two paths are compared along five dimensions and shown to be complementary: the first suited to operational risk monitoring within bounded deployments and the second to the governance architecture that determines which agents may legitimately be deployed at all.

Read PDF

Similar papers

Review Oct 2026

Business Compliance and Agentic AI Liability

Agentic AI systems are being deployed at scale in enterprise environments, yet the legal framework for determining who bears responsibility when these systems cause harm remains fundamentally unresolved. This article argues that agentic AI produces a categorically distinct form of harm called trajectory-based harm, w...

Marcel Osmond · 0 citations
Sep 2026

Anticipatory Human Oversight of Agentic AI: A Philosophical Account

This paper argues that reactive oversight must be complemented by an anticipatory mode: oversight exercised before the agent acts, by specifying the normative agenda that structures the space of permissible action and refining it iteratively through specification, runtime, and inspection.

Kevin Baum, Maximilian Kiener, Markus Langer et al. · 3 citations
#artificial intelligence Preprint Oct 2026

Agent Reliability Profiles in Financial Services

AI agents can take actions. At times, those actions can go beyond what is intended. Agent reliability can be defined as assurance that an agent will stay within intended bounds and operate within limits. Today, there is no shared framework or language for describing, validating, and benchmarking the reliability of agen...

Mike Hsu, Medha Bankhwal, Béatrice Moissinac et al. · 1 citation
Sep 2026

Responsibility to the Problem: An Explicit Standard for the Expert–AI System

Discussions of artificial intelligence in science ask who is responsible when AI produces an incorrect result. This question is vague unless it identifies the object of responsibility, evidence, decisions, and cognitive process. Public criticism often isolates an erroneous output while omitting the question, informatio...

Laboratory of Information Systems · 0 citations
Sep 2026

Preliminary Reflections on the Legal Governance of Agentic AI: A Private- and Public-Law Reconstruction through the Permission–Action–Oversight–Liability Framework

Artificial intelligence has moved beyond the generative stage—producing outputs and proposing decisions to humans—toward an agentic stage, in which systems autonomously pursue delegated goals and act directly upon external systems. Yet the prevailing ‘risk-based’ approach—the high-risk classification of the EU AI Act a...

Jungmi Bang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.