Judgment Cannot Be Delegated: A Subject-Preserving Framework for AI Governance
TL;DR
It is argued that algorithmic assistance can legitimately expand human deliberation, whereas delegation dissolves the very subject who judges, and derived governance theorems for non-delegability, contestability, reversibility, and subsidiarity.
Abstract
Contemporary institutions increasingly treat optimised procedures as decisions in their own right. This article advances an ontological limit claim for AI governance: moral judgment is constitutively personal and therefore non-delegable. Building on a minimal philosophical anthropology—person/thing distinction; irreducibility of phronēsis; the person as an end; responsibility as constitutive; and the capacity to initiate—we argue that algorithmic assistance can legitimately expand human deliberation, whereas delegation dissolves the very subject who judges. We situate the claim within current debates on artificial agency and Meaningful Human Control (MHC), and show how a subject-preserving reading supplements tracking/tracing by specifying what must remain human in dignity-touching domains. Two diagnostic cases—criminal-justice risk scoring and AI-steered coverage decisions in healthcare—illustrate a structural tendency to displace judgment by optimisation; they are not offered as empirical proof but as paradigmatic contexts where institutional deference to model outputs risks rendering answerability merely nominal. From our axioms we derive governance theorems for non-delegability, contestability, reversibility, and subsidiarity, and we close with an implication for the institutional formation of those who exercise judgment, which exceeds the scope of this article and is taken up in a companion paper. The result is a framework that welcomes instrumental progress while marking a principled boundary: systems may optimise for us; they may not judge instead of us.