Leakage-Controlled Cross-Dataset Evaluation of Self-Supervised Graph-Based Network Intrusion Detection

Abstract

Flow-based network intrusion detection systems (NIDS) report steadily improving benchmark results, yet these results are frequently inflated by feature leakage and by evaluation confined to a single dataset. This article turns leakage from a declared precaution into a measured quantity and packages the measurement as a reusable evaluation protocol: a feature policy that removes IP addresses and reduces ports to three semantic categories, supervised probes that quantify each suspected shortcut channel, mandatory zero-shot cross-dataset testing, and base-rate-aware reporting. A port-only probe separates the classes almost perfectly inside NF-BoT-IoT-v2, at ROC-AUC 0.998, but falls to 0.615 on NF-UNSW-NB15-v2 — the gap is the size of the shortcut. The protocol is exercised end to end with FlowGuard, a lightweight self-supervised graph autoencoder trained exclusively on benign traffic. Under the sanitized protocol FlowGuard reaches ROC-AUC 0.968 on NF-UNSW-NB15-v2; raw cross-dataset transfer collapses to 0.332, and an unsupervised CORAL alignment step restores it to 0.926 without any target-domain labels, subject to a benign-majority condition that we characterize and operationalize with three label-free indicators. The model has 32,617 parameters (0.13 MB), processes one flow in 1.48–1.69 microseconds on four CPU threads, and its decision threshold is calibrated without labels: thresholds set on training benign traffic yield test false-positive rates of 0.0098, 0.0247, 0.0495 and 0.0987 for targets of 0.010, 0.025, 0.050 and 0.100. We also report negative results, including an IP-memorization probe that did not collapse under grouped splitting, and release the complete pipeline for replication.