This paper argues that reactive oversight must be complemented by an anticipatory mode: oversight exercised before the agent acts, by specifying the normative agenda that structures the space of permissible action and refining it iteratively through specification, runtime, and inspection.
Abstract
Human oversight is widely held to mitigate the risks of AI systems. Even for systems that produce discrete outputs at identifiable decision points, the realisation of human oversight as a reactive measure is empirically fragile, yet increasingly well understood. However, for agentic AI -- systems that plan, decompose goals, and execute multi-step actions over extended horizons -- reactive oversight reaches its structural limits: intervention on individual actions defeats the autonomy that motivates the deployment, while intervention on aggregate patterns is too coarse for harms whose cumulative consequences only become legible after the fact. This paper argues that reactive oversight must be complemented by an anticipatory mode: oversight exercised before the agent acts, by specifying the normative agenda that structures the space of permissible action and refining it iteratively through specification, runtime, and inspection. The two are complements -- the agenda's escalation conditions specify when reactive intervention is invoked. Drawing on Meaningful Human Control, we read anticipatory oversight as the operationalisation of distal-reason tracking. In addition, we argue that the proposed framework yields a specific responsibility architecture by design: occupying the anticipatory mode is the discharge of a role-grounded prospective obligation, and backward-looking responsibility takes the form of strict moral answerability -- rationalistic, relational, and holding regardless of fault, in virtue of the principal's prior opportunity for precaution. We develop bridging failure modes, address objections including moral luck and the illusion of control, and close with regulatory, architectural, and empirical implications
It is concluded that only by grounding oversight in realistic assumptions about human cognition and institutional incentives can it fulfill its intended role in ensuring trustworthy AI.
A. Henestrosa, A. Grinbaum, Ivan P. Yamshchikov· 0 citations
Two paths for conceptualising agentic risk are developed: a conservative path, which retains the standard definition of risk and reconstructs it formally for narrow agents; and a governance path, which draws an analogy to the normative systems used to manage biological agents, arguing that sufficiently general AI agent...
Federico L. G. Faroldi· European Journal of Risk Reg...· 0 citations
Human oversight is widely prescribed for high-impact artificial intelligence, but “human in the loop” is not an operational policy when review capacity is finite and agent activity varies over time. We formulate oversight for financial AI agents as a capacity-constrained allocation problem: for each proposed action, a...
Personal AI agents are increasingly capable of handling long-horizon tasks across applications, yet users often need to invest substantial effort in specifying goals, resolving ambiguity, and verifying outcomes. We argue that progress should be measured by how effectively agents amplify human agency: enabling people to...
Agentic AI systems are deployed to execute complex tasks across domains such as writing, coding, and data analysis. As execution is offloaded to AI agents, the human role shifts toward oversight. Effective oversight requires recognizing when closer scrutiny is needed, even though flawed outputs may appear convincing. O...
Dinithi Dissanayake, Yasith Samaradivakara, Hyung Won Lee et al.· 0 citations
The rapid deployment of agentic AI has introduced complex challenges for the ethics and governance of AI systems. Agents built on large language models act through tools on behalf of people, and each deployment case brings its own instructions, policies, evidence, and stakeholders, so alignment must be formed case by c...
Quan-Yan Zhu· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.