mic-plan benchmark protocol
Abstract
This record registers, before any power or classification was computed, the protocol and the complete code of a benchmark of the Python package mic-plan 1.0.0 against ten sample-size methods for studies that compare minimum inhibitory concentration (MIC) distributions. The methods are stats::power.prop.test, Hmisc::posamsize (Whitehead 1993), samplesize::n.wilcox.ord (Zhao et al. 2008), WMWssp (Happ et al. 2019), pwr::pwr.chisq.test with Cohen's w, stats::power.t.test on mean log2 MIC (after Zawack et al. 2016), simulation at the pilot proportions, posterior-average power, and the 90th percentile of the Zhao et al. size over pilot bootstrap resamples and over Dirichlet posterior draws. Every plan is judged against the true power of the test it implies. The validation set holds every 2021 EFSA stratum of indicator Escherichia coli with at least 150 isolates and every 2021 FDA NARMS cecal E. coli stratum with at least 150 isolates for ampicillin, chloramphenicol, ciprofloxacin and tetracycline (130 distributions, three changes each, 20 pilots of 20 isolates); no computation of the project had used these distributions. The development sets hold the 2023 distributions used while mic-plan was built and in an earlier six-method comparison (doi:10.5281/zenodo.22997395). Hypotheses: H1, mic-plan leaves at most 10% of its validation plans below 80% power; H2, fewer than each of the eight published methods (Holm adjustment); H3, a positive control for four point-estimate methods. BENCHMARK_PROTOCOL.json gives the laws, changes, methods, analysis tests, truth calculation, endpoints, hypotheses and decision rules (protocol digest 35a3e62ace71ab8ca0bc327b32a27fcb781324880998f5c2f4dc75af071436e9). SIZE_CHECK.json shows that the six formula-based sizes agree with the R packages on all 23,580 pilots; before this deposit only sizes had been computed, apart from a software check that also estimated three power points of one validation law (law 519 at 160 isolates per group for the rank, chi-square and t-tests); these were not classified or used.
