Neither too conservative nor too liberal: routing R×C tables of independence to the most efficient valid test (m02e) -- Reproducibility deposit

Abstract

Neither too conservative nor too liberal: routing R×C tables of independence to the most efficient valid test William J. Dwyer, MD, MPH, FAAP — Department of Mathematics and Statistics, University of Massachusetts Lowell, Lowell, MA, USA. ORCID 0009-0004-0855-7222. License: MIT for all code; CC BY 4.0 for documents, figures, and data. What this is The reproducibility package for the R×C generalization of the 2×2 achieved-size study (the sibling 2×2 paper, Dwyer 2026). It carries one question to general tables of independence: when is the "exact" test least exact, and when is the χ² default most wrong? The package computes, per test and per design, the achieved (unconditional) size — the supremum over the product-multinomial null nuisance of the rejection probability — by exact enumeration where the design is enumerable, and by a bias-aware calibrated Monte-Carlo backend where it is not, with the exact backend as the credibility anchor for the Monte-Carlo path. Nothing here is a black box: every number is produced by a named script, the exact results carry no simulation, and the Monte-Carlo figures are seed-fixed and reported with standard errors. The keynote The verdict a table receives depends on which test is used, precisely on the decision line, and that region grows with table size. On 3,886 real contingency tables cross-classified from 757 public pydataset/Rdatasets datasets (no simulation), the roster agrees essentially everywhere away from the cutoff but disagreement spikes on it: 79% of tables within ±0.02 of α = 0.05 receive a test-dependent verdict, and the disagreement runs both ways — the asymptotic χ² family over-rejects, the negative-λ divergences (Neyman, modified G²) under-reject. Real-data direction is read against the valid band — the tests valid at every design (FFH and its Boschloo raise) — not against any single "frontier": a test over-rejects when it rejects where no valid test does, and under-rejects when it misses what all valid tests catch. Validity is a size property, judged from the achieved (sup) size and kept strictly separate from real-data direction; the manuscript's definitions block fixes this vocabulary (validity as a size property; the randomized size-α oracle as a reference, not a deployable test; the valid band as a set with no single winner). The findings The honest gap opens on both sides at once and widens with the table: the exact conditional test (Fisher–Freeman–Halton, FFH) grows more conservative as cells multiply, while the asymptotic χ² family grows more anticonservative as the minimum expected count thins. Matched to a common size, the χ² family's apparent power advantage is revealed as borrowed size — on a 3×3 design Freeman–Tukey's nominal power 0.54 collapses below the exact test's once its rejection region is made valid — and every valid test then sits a measured distance below the randomized size-α oracle, that distance being the recoverable power. The package closes with a routing rule keyed to table shape and sparsity and an in-browser demonstrator. The Boschloo–FFH band member The valid band includes a size-spending unconditional-exact member, Boschloo–FFH: Boschloo's (1970) size-raise applied with the FFH conditional p-value as the ordering statistic and the supremum taken over the product-multinomial nuisance, deployed for general R×C. It is valid by construction and dominates FFH. This is an obvious composition of two classical ingredients (Boschloo 1970; Freeman–Halton 1951) and, to the author's knowledge, has not previously been published in this form for R×C tables; the nearest precedent is the 2×c unconditional exact test of Ehwerhemuepha, Sok and Rakovski (2019), which uses a simulation-and-regression threshold rather than the exact supremum raise. It is presented as a deployable band member, not a new test statistic. What this paper adds Following the house novelty standard (a lead + Strand / Prior art / What this paper adds / Degree of novelty, the house novelty-block organization), and mirroring the manuscript's §10 "Relation to prior work and the novelty boundary": Strand. The honest-reporting program for contingency-table tests: what a test's achieved size and power are (not its nominal ones), how validity and power trade off, and which test to deploy where. This is the R×C sequel to the sibling 2×2 study (Dwyer, 2026), which it nests and extends. Prior art (claimed as none new). That FFH is conservative and the χ² approximation degrades at small expected counts is old (Yates 1934; Cochran 1954; Larntz 1978); the exact conditional R×C test is Freeman and Halton (1951), efficiently computed by the network algorithm of Mehta and Patel (1983); the uniformly-more-powerful unconditional route at 2×2 is Boschloo (1970) with validity by nuisance maximization (Berger and Boos 1994); the power-divergence family and its small-sample behavior are Cressie and Read (1984) / Read and Cressie (1988); the modern recommended-tests program is Lydersen, Fagerland and Laake (2009) and Fagerland, Lydersen and Laake (2017); the landscape is surveyed by Agresti (2001). What this paper adds. (1) The characterization and the rule — a single deterministic achieved-size-and-power object across the R×C shape × sparsity space, showing the honest gap opens two-directionally (FFH more conservative as cells multiply; the χ² family more anticonservative as the minimum expected count thins) and widens with cell count — the 2×2 story was one-directional; the R×C story is not. (2) Boschloo–FFH, the one construction that is not simply a citation: Boschloo's size-raise applied with the FFH conditional p-value as the ordering statistic and the supremum over the product-multinomial nuisance, deployed for general R×C — valid by construction, dominates FFH, and, to our knowledge, not previously published in this form for R×C (nearest precedent: the 2×c test of Ehwerhemuepha, Sok and Rakovski 2019, a simulation-and-regression threshold for the two-row case only). (3) Borrowed size, made measurable — the χ² family's apparent power edge, benchmarked against the randomized size-α oracle, is shown to be borrowed size that evaporates under size-matching. (4) A calibrated Monte-Carlo backend gated to the exact truth (within 3.5 MCSE on a 28-cell matrix), carrying the surface past the combinatorial wall, plus a real-corpus R×C verdict-flip scan (3,886 tables) and a shape-and-sparsity routing rule with a parity-checked demonstrator. Degree of novelty. The critique and the remedies are classical and cited in the same breath; the novel deliverables are the R×C achieved-size-and-power characterization, one deployable band member (Boschloo–FFH — an obvious composition of two classical ingredients, but not previously published for R×C), and the empirical at-scale and real-corpus demonstrations. The novelty line against the author's own adjacent work is explicit: the sibling 2×2 study (Dwyer, 2026) is the special case this paper nests and extends. The pre-empted referee objection. A reviewer's natural challenge — "isn't a well-chosen asymptotic statistic fine?" — is met head-on: even Cressie–Read λ = 2/3, the best-calibrated power-divergence member, does not escape R×C discreteness, so the paper scopes the "χ² default overshoots" claim to the common defaults (Pearson, G², Freeman–Tukey, Neyman) and shows the exact unconditional route is what closes the gap. The band-not-frontier discipline (no single winner among valid tests) is the guardrail that keeps the argument honest rather than promotional. What the deposit contains rerun/ — the exact R×C engine (rc_exact.py, rc_roster.py), the Boschloo–FFH construction (rc_boschloo.py), the calibrated Monte-Carlo backend (mc_backend.py), the achieved-size / power / sweep / real-corpus scripts, and the JS demonstrator engine (rc_engine.js) with its parity fixtures. The credibility gates, all passing: rc_parity_gate.py / rc_roster_parity.py (the 2×2 reduction reproduces the sibling paper's deposited exact sup-sizes for FFH, mid-p, and Pearson, 0 mismatches); mc_gate_extended.py (the bias-aware Monte-Carlo sup reproduces the exact backend within 3.5 Monte-Carlo standard errors across a 28-cell shape × test matrix); rc_boschloo.py (validity + FFH domination on every shape, and the 2×2 achieved sizes reproduce the sibling paper's Table 1 Boschloo column to ≤0.005). figures/ — the six manuscript figures (size vs sparsity; size-adjusted power vs the oracle; two- directional error; routing map; at-scale Monte-Carlo surface; real-corpus flips). The documents — manuscript, long-form derivations companion, supplement, and the Statistica Neerlandica cover letter — and honest_rc.html, the interactive demonstrator (its engine verified against the Python roster to five decimals). README.md and a MANIFEST.sha256 of every file. Credibility anchor The 2×2 credibility anchor is the deposited sibling-paper package (Dwyer, 2026), concept DOI 10.5281/zenodo.22238930— the exact backend's 2×2 reduction is gated against its deposited exact values. How to cite Please cite this deposit if you use the package or the method. Citing the concept DOI references the work in general and always resolves to the latest version; cite a specific version DOI to point at an exact snapshot. Dwyer, W. J. (2026). Neither too conservative nor too liberal: routing R×C tables of independence to the most efficient valid test — reproducibility deposit [Software]. Zenodo. https://doi.org/10.5281/zenodo.22178475 BibTeX: bibtex @software{dwyer_m02e_2026, author = {Dwyer, William J.}, title = {Neither too conservative nor too liberal: routing RxC tables of independence to the most efficient valid test --- reproducibility deposit}, year = {2026}, publisher = {Zenodo}, doi = {10.5281/zenodo.22178475}, url = {https://doi.org/10.5281/zenodo.22178475}, orcid = {0009-0004-0855-7222} } The DOI above is the conc