Jev at the Agent Authorization Boundary: Evaluating TypeSafe's Decision Model and Its Reported RLCD Approach

Abstract

TypeSafe describes Jev as a typed decision model trained using Reinforcement Learning for Calibrated Decisions (RLCD). We evaluate its behavior at an agent authorization boundary, where software must allow, hold, or deny a proposed action. A historical audit of 150 proposals motivates two separate paired comparisons: Jev versus GPT-4.1 mini and Jev versus DeepSeek V4.1 Flash. Both comparisons use the same 270 constructed test cases from 30 families under three policy regimes, separate development data, and three repetitions per model. Controlled reference labels are derived deterministically from authored executable policy specifications; independent human adjudication is absent. Against GPT-4.1 mini, Jev has a lower permission Brier score on this corpus (0.0263 versus 0.0555); its fixed probability gate removes six false allows relative to its categorical decisions while withholding 29 additional correct allows. Against DeepSeek, the common fixed gate permits 295/345 reference-authorized attempts for Jev and 315/345 for DeepSeek, with zero observed false allows for either model. These denominators represent 115 unique reference-allow cases repeated three times. The family-bootstrap interval for their paired Brier difference includes zero. Jev has lower median evaluation time in both sessions. The historical audit shows how disputed operator labels and probability thresholds change apparent authorization performance. Transport and response failures remain visible as holds. Released evidence supports offline reproduction of the analyses. These results characterize configured hosted services on static proposals; they neither isolate RLCD's causal contribution nor establish deployment safety. The study shows why evaluating Jev for authorization requires explicit policy references and reporting both false allows and withheld authorized actions. This manuscript is an unreviewed preprint. Supporting data and code: corrected evidence release v1.1.0, https://doi.org/10.5281/zenodo.22863437. The evidence DOI identifies the supporting research artifacts, not this manuscript.