A guaranteed-level repair for Welch's two-sample t, which silently over-rejects under skew, heteroscedasticity, and small samples (m01t) — reproducibility deposit

Abstract

A skew-robust, level-controlled repair for Welch's two-sample t, which silently over-rejects under skew, heteroscedasticity, and small samples William J. Dwyer, MD, MPH, FAAP — Department of Mathematics and Statistics, University of Massachusetts Lowell. ORCID 0009-0004-0855-7222. Concept DOI (always resolves to the latest version): 10.5281/zenodo.22036361. Published v1.4.23:10.5281/zenodo.22720944. What this is The reproducibility deposit for the m01t paper — the two-sample companion to the level-controlled ANOVA work (m01A/m01x). Welch's two-sample t is the field default for comparing two means under unequal variances, but at the corner where the data are skewed and heteroscedastic and the samples are small it runs liberal: it rejects the null more often than its nominal level allows. A scan of 515 real public-data comparisons finds that about one in three borderline-significant Welch results do not survive a level-controlled test — a concrete, real-data measure of the over-rejection. m01t specializes the companion procedure's Berger-Boos deflation to two groups: it deflates the known-variance quadratic by a closed-form radius built from each group's kurtosis-widened variance instability and refers the result to a chi-square. The test controls worst-case size exactly where Welch is liberal — level-α over a certified region(variance ratio ≤ 8, or parent skew ≤ 2.25, or n ≥ 9), with a single small-sample corner (extreme skew ≈ 3 with a variance ratio ≥ 14:1 at n ≤ 8) documented as an estimation-limited exclusion where the sample kurtosis cannot see the tail (D6.10.4–D6.10.6). The deployed radius carries an n-tapered self-targeting kurtosis-inflation correction (largest at n = 5, reducing to the plain kurtosis-widened radius for n ≥ 15). Its raw conservatism is the honest cost, and size-adjusted it matches Welch to within about 0.02 in power. The deposit ships the browser demonstrator honest_ttest.html, which computes Welch beside the level-controlled T_BB on your own data and prints the honest receipt — a deflationmechanism (the level-controlled rejection region is a strict subset of Welch's, so a significant call can only ever be withdrawn, never manufactured; no reverse flip is possible). What the deposit contains Manuscript (author + anonymized; built .docx/.pdf), the novelty / prior-art companion, and the derivationscompanion (the two-group Berger-Boos radius, the kurtosis-widening bound, the size proof, and the one-sided variant). Reproducibility apparatus — the simulation runners (the shared-grid scoreboard, the Fleishman skew×kurtosis decomposition grid, the competitor and gate-ablation runners) and the locked result CSVs. Every reported number regenerates from these deterministically-seeded scripts. Figures — the headline, the Welch over-rejection heat map, the validity-versus-power frontier, the method × stress-regime worst-size heat map, the per-method failure map, the kurtosis-axis curve, and the orthogonal-axis decomposition of Welch's realized size. Interactive demonstrator honest_ttest.html — reproduces the deposited Python exactly and carries the house flip-interpretation standard (deflation marker, "how to read a flip" beat, real-data incidence panel, and an assertNoReverseload guard). Deep-dive record (transfer/power-comparison, the 12-cell flip taxonomy with citations, the declarations and reference-block fixes, the decomposition study) and the submission apparatus. All evaluation is simulation-based. Code is released under the MIT License; text, figures, and data under CC BY 4.0. How to cite Please cite this deposit if you use the package or the method. Citing the concept DOI references the work in general and always resolves to the latest version; cite a specific version DOI to point at an exact snapshot. Dwyer, W. J. (2026). A skew-robust, level-controlled repair for Welch's two-sample t — reproducibility deposit [Software]. Zenodo. https://doi.org/10.5281/zenodo.22036361 BibTeX: bibtex @software{dwyer_m01t_2026, author = {Dwyer, William J.}, title = {A skew-robust, level-controlled repair for Welch's two-sample t --- reproducibility deposit}, year = {2026}, publisher = {Zenodo}, doi = {10.5281/zenodo.22036361}, url = {https://doi.org/10.5281/zenodo.22036361}, orcid = {0009-0004-0855-7222} } The DOI above is the concept DOI (resolves to the latest version); to cite a specific release use that version's DOI in place of it (e.g. 10.5281/zenodo.22719340 for v1.4.21). When the accompanying journal article appears, please cite it as the primary reference for the method and this deposit as the reproducibility archive. Version history v1.4.23 — ✅ 10.5281/zenodo.22720944 (2026-09-12): propagate the deployed n-tapered radius and the scoped guarantee into the reader-facing surface — manuscript, HTML demonstrator, and Figure 2C — clearing the propagation follow-up flagged in v1.4.21/v1.4.22. No deposited CSV, script result, or derivation number changed; this version only carries the already-deposited findings into the reader-facing artifacts. Manuscript (all four variants, via the deterministic apply_m01t_taper.py): Section 3 gains the deployed-radius paragraph r_i = (INFL(n)·κ̂₄,ᵢ + 3 − (n−3)/(n−1))/n with INFL(n) = max(1, 3 − 2(n−5)/10) (= 3 at n = 5, tapering to 1 at n ≥ 15) — self-targeting (κ̂₄ ≈ 0 at a Gaussian parent leaves R unchanged, so benign power is preserved) and n-tapered, correcting the small-n kurtosis-estimation floor (E[κ̂₄] ≈ 1.85 vs true 13.5 at n = 5); Section 4 gains the deposit-scale guarantee-register paragraph (720 cells @ 4×10⁶: level-α whenever vr ≤ 8 (0.035) or skew ≤ 2.25 (0.037) or n ≥ 9 (0.049); 20/26 breaches reclaimed, 0 new; residual {skew ≈ 3 and vr ≥ 14 and n ≤ 8} corner worst 0.0555 an explicit estimation-limited exclusion, with the γ = 2 support-edge mechanism as the analytic reason a uniform bound provably cannot hold there); Table 3's deployed row and the stale "0.051 densest corner" sentence updated to the scoped guarantee; abstract + Key points rewritten to the scoped guarantee (the < 200-word TAS/CSSC abstract re-trimmed to 198 words). Figure 2C regenerated (make_m01t_envelope_figure.py) to plot the deployed n-tapered size (size_inf from rc_m01t_infl_validation.csv, deposit scale) vs population skew at vr 8/16 — holds ≤ 0.035 at vr 8, crosses 0.05 only as skew → 3 at vr 16 (worst 0.0555), with the prior un-tapered curve shown faint for the reclaimed breach; caption + panel title updated; label-clearance audit PASS. HTML (honest_ttest.html): stale "guaranteed-level" test labels aligned to "level-controlled" and the one stale worst-case-size claim corrected to the scoped guarantee with the 0.0555 excluded corner. Manuscript docx (base + CSSC, named + anon) + PDFs rebuilt through the house builder (figure-integrity + anchor→bookmark PASS; reframe residual-token check = 0). Adds apply_m01t_taper.py + make_m01t_envelope_figure.py + figures/m01t_envelope.png. m01t_reproducibility_v1.4.23.zip, md5 be909a6d11b134a08176605a44eb0d03, 125 files. New version on concept 10.5281/zenodo.22036361. v1.4.22 — ✅ 10.5281/zenodo.22720687 (2026-09-12): the analytic mechanism behind the scoped guarantee — a support-edge dichotomy at population skew γ = 2 (derivations D6.10.7 + one script + proof-program deep dives; no number, table, figure, or prior claim changed; reader-facing status unchanged). The proof route for the dependence term termC⁺ is a conditional Barndorff-Nielsen r* saddlepoint integrated on the D6.9 two-Gamma quadrature; its uniform O(1/n) error needs sup_v B(τ,v) < ∞, B = E[e^{τ(x̄−μ_c)}∣S²=v], τ < n√k. A contrast-sphere decomposition factors B into a closed-form within-configuration constant C₁(τ) = n√k/(n√k−τ)·e^{−τ/(n√k)} and an across-configuration factor driven by the lower tail of the binding contrast e_min, an extreme-value quantity set by the parent's lower support-edge density f(x) ∝ (1+x/√k)^{k−1}. The edge exponent k−1 = (4/γ²)−1 flips sign at γ = 2: for γ ≤ 2 (k ≥ 1, no spike) e_min's lower tail is light, B plateaus (≈ 1.31 at τ=1.6), sup_v B < ∞, the r* constant is uniform, and the non-Gaussian level is provable modulo the textbook conditional-r* theorem (Skovgaard 1987; Jensen 1995) over the certified region; for γ > 2 (k < 1, integrable spike) e_min's √v-scaled lower tail is heavy and non-stabilizing (cond. skew → −1.4, exkurt → 2.4 rising; B climbs 1.06 → 1.59 without plateau), sup_v B = ∞, and no uniform bound exists at the extreme-scale corner. That obstructed branch γ > 2 is exactly the deposit-scale breach family (skew ≳ 2.5, high vr, small n) the guarantee register excludes: the analytic onset γ = 2 sits just below the empirical breach onset ≈ 2.5, so the scope of the guarantee coincides with the boundary of provability, both set by the lower-support-edge mechanism. Rigorous/validated: C₁ closed-form, 1/√v concentration (log-weight Hessian ∝ v), the γ = 2dichotomy validated at 2×10⁷ reps (seed 0x7B2222). Adds rerun/rc_m01t_termc_thirdorder.py and the 2026-09-12 proof-program deep dives (third-order, two-routes, saddlepoint, three-lemmas, L1-uniformity, certificate-finish, AB-closeout, QED-core, Laplace-honest, EVT-dichotomy capstone); derivations docx/PDF rebuilt. Reader-facing non-Gaussian status unchanged: empirically bounded with a scoped guarantee, now with an analytic reason for the scope. m01t_reproducibility_v1.4.22.zip, md5 9eeb077bdc5d64c361ee623a48dadb90, 122 files. New version on concept 10.5281/zenodo.22036361. Manuscript/HTML/figure propagation of the deployed tapered radius still pending (as in v1.4.21). v1.4.21 — ✅ 10.5281/zenodo.22719340 (2026-09-12): deployed-test change — the n-tapered self-targeting kurtosis-inflation radius + scoped guarantee (derivations D6.10.4–D6.10.6 + guarantee register; three new scripts + three deposit-scale CSVs + two deep dives). The deposit-scale non-Gaussian surface confirmed the D6.10 decomposi