Prospective Worst-Attack Reliability Floors for Flow-Based Machine-Learning Intrusion Detection under Forward-Time Shift

Abstract

Flow-based machine-learning intrusion detectors are commonly assessed using aggregate scores, which can conceal weak attack-family recall. This study tests a source-calibrated minimum-recall floor on a short forward-time UNSW-NB15 capture as a proof of methodology. Four raw files were deduplicated to 2,059,414 flows and divided into group-disjoint source partitions and a later same-environment test. Logistic regression, random forest, and XGBoost five-seed ensembles were screened using source data only. The contribution integrates family-recall calibration, simultaneous lower confidence bounds, frozen retention criteria, and cryptographic prediction sealing before future-label evaluation. On 1,034,783 future primary flows, the locked XGBoost ensemble achieved 95.9% accuracy, 68.1% balanced accuracy, a Macro F1 of 0.599, and an MCC of 0.773. Its minimum simultaneous recall bound was 13.3%, narrowly exceeding the frozen 12.6% floor, while worst-family point recall was 22.0%. Analysis, Backdoor, and Reconnaissance did not retain their family-specific bounds. Post-review checks preserved global retention at 5,000 and 10,000 bootstrap replicates, but alternative temporal-block groupings did not consistently yield a future bound above the original floor. Feature diagnostics showed both missingness and observed-value changes without establishing causality. XGBoost remained best among the three fixed configurations. Global floor retention therefore supports a limited protocol-conditional finding; it does not establish acceptable detection performance, uniform family stability, or contemporary deployment readiness.