Judgment Cost Theory, Study 2 — Replication and Audit Archive: When Automation Concentrates Judgment: A Systematic Review and Meta-Analysis of Residual Decision Difficulty and Oversight Burden Following AI Adoption

Abstract

Version 2.3 (21 September 2026). No content changed between 2.2 and 2.3. Every file, hash and result is identical. This version exists only to correct the file list: version 2.2 was published carrying both the superseded v2.1 archive and the current one, because Zenodo copies prior files into a new-version draft by default and published files cannot afterwards be withdrawn. Version 2.3 ships a single archive. Versions 2.1 and 2.2 remain published and citable at their own DOIs; this is a correction of presentation, not a withdrawal. Replication and audit archive for Judgment Cost Theory, Study 2: When Automation Concentrates Judgment: A Systematic Review and Meta-Analysis of Residual Decision Difficulty and Oversight Burden Following AI Adoption (Jason West, Liberty University). The review tests the theory's concentration and non-substitution propositions against 80 real-world deployments of learned decision-support systems, identified from 17,120 records across thirteen sources, with screening and extraction performed in duplicate by paired large-language-model reviewer-agents under frozen prompts and every disagreement adjudicated by the author. The archive is the complete and corrected record of the review: the registered protocol and dated amendment log, coding manual (v2.3) with superseded versions, search log and database strings, reliability ledger with blinded recall audits, frozen reviewer and coder instruments, per-record screening dispositions, corrected PRISMA 2020 flow, the 80-study analytic datasets with a replication codebook, all analysis scripts with a verification routine (run_all.py --verify) that reproduces every deposited result exactly, manuscript tables and figures, and a consolidated deviations-and-corrections list (X1–X12) that documents every departure from the registered plan and every correction of the record, including a post-deposit source re-verification of all computable effect sizes. A SHA-256 manifest covers every file. Requirements for reproduction: Python 3 with NumPy and SciPy. This archive accompanies the OSF registration at https://doi.org/10.17605/OSF.IO/8YB9G (submitted 3 August 2026, issued 6 August 2026). The registration is frozen at submission and records the protocol and screening trail as they stood before extraction; where the two differ, this archive is authoritative for everything dated after 3 August 2026, and the deviations list explains every difference. This is v2 of the archive, superseding the 6 August v1 build; the README's opening section explains what changed and why. Version 2.2 (21 September 2026). This version adds the structured narrative synthesis of the non-poolable evidence, which the protocol specifies for outcomes that cannot be pooled and which earlier versions did not contain. Its absence was found by a drift audit on 18 September 2026 and reinstated by program decision P21. No pooled estimate, PRISMA count, risk-of-bias appraisal or registration record changed. New in this version: the narrative synthesis script and its results, registered in the verification harness; two harvest plots generated by that script; a pinned requirements file recording the computational environment; the drift audit and its dated correction; the program decision log; the drift audit protocol the audit was run under; and the manuscript builder, which now parses figure captions from the manuscript rather than holding its own copies. Changed in this version: the manuscript in all three forms, now carrying the narrative synthesis, H3 retitled from "not testable" to "not estimable by meta-regression", confidence intervals on every agreement statistic, exact counts of what the post-deposit correction changed, the reviewer-agent model family and run dates with an explicit statement of what was not recorded, a table separating pre-specified from post hoc analyses, and the Zenodo citation corrected from a version DOI to the concept DOI. The amendment log gains a dated reconciliation recording that the stratified learned-model resolution did not restore the accountability contrast, against its original claim that it did. The deviations list gains X14, which records that five of the six deposited result files reproduce exactly while one differs on two printed values in a rank-deficient cell where no estimate exists. Archive contents: 150 files with a SHA-256 manifest. Every file from version 2.1 is carried forward, 132 of them byte-identical.