A meta-review of 18 studies between 2019 and 2026, from which 83 ICS, or ICS directly related, cybersecurity datasets are identified, harmonised, and characterised using a unified five-dimensional taxonomy, identifies three structural imbalances which constrain the scope and feasibility of several evaluation practices.
Abstract
Intrusion detection research in Industrial Control Systems (ICS) heavily depends on public datasets, yet no prior work has systematically assessed whether the collective dataset corpus supports current evaluation claims. This paper addresses this gap through a meta-review of 18 studies between 2019 and 2026, from which 83 ICS, or ICS directly related, cybersecurity datasets are identified, harmonised, and characterised using a unified five-dimensional taxonomy. The taxonomy reveals that the corpus is structurally skewed: 85.5% of datasets concentrate on late-stage OT Disruption tactics, cross-stage IT/OT progression sequences are present in only 8.4% of cases, field-device evidence at Level 0 of the Purdue hierarchy is effectively absent, and operationally sourced data accounts for only 15.7% of the collection. A parallel audit of evaluation practices shows that zero report streaming evaluation, fewer than half apply disciplined train/test partitioning, and only two satisfy reproducibility requirements. Furthermore, a taxonomy-evaluation coupling analysis shows that dataset imbalances constrain the scope and feasibility of several evaluation practices. Based on these findings, we identify three structural imbalances: i) architectural shallowness, ii) progression compression, and iii) cross-domain substitution, and derive a coordinated research agenda which covers cross-stage corpus construction, temporally structured benchmarking, event-level label standards, and governance frameworks for operational data sharing.
Industrial Control Systems (ICSs), which underpin communications and operations in industrial environments in general and in critical infrastructures in particular, are targeted by malicious actors for diverse reasons. To strengthen their resilience against cybersecurity attacks, testbeds play a fundamental role in supporting experimentation, the validation of defense mechanisms, dataset generation, and architecture evaluation, among other activities. However, existing testbeds differ considerably in their functional capabilities, architectural completeness, implemented technologies, and degree of documentation, making systematic comparison and classification difficult. This paper presents a comparative analysis of 34 ICS cybersecurity testbeds within a Purdue-aligned framework based on two complementary dimensions: Functional Maturity (FM) and Architectural Completeness (AC). The former evaluates capabilities relevant to cybersecurity experimentation, while the latter measures the coverage of Purdue-aligned and supporting IT/OT components the testbed includes. To address this problem, we propose a methodology that enables the identification of technology adoption patterns, capability distributions, and recurring architectural characteristics across contemporary testbed implementations. Results reveal that many platforms provide advanced experimentation capabilities while offering only partial representation of industrial and enterprise environments. Moreover, the analysis also highlights underrepresented architectural components that may influence the scope of cybersecurity, Industrial Internet of Things (IIoT), and Industry 5.0 research that can be conducted within a testbed. Finally, based on these findings, a Purdue-aligned testbed blueprint is proposed as a reusable conceptual baseline for the design and comparison of future industrial cybersecurity experimentation environments.
Jovan Andrés Guillén-Mass, Roberto Magán-Carrión· Electronics· 0 citations
To improve cybersecurity across industries, Cyber Threat Intelligence (CTI) is becoming increasingly crucial. This systematic review explores how CTI practices are evolving in response to advancements in Artificial Intelligence (AI), particularly in the context of Large Language Models (LLMs). We examined 61 peer-reviewed studies using the PRISMA methodology, which demonstrates a strict selection procedure founded on specified inclusion, exclusion, and quality standards. This approach aligns with the scope of similar systematic reviews in the field of cyber threat intelligence. The review provides a comparative synthesis of CTI research capabilities across threat detection and prediction, attribution, forecasting, and automated reporting. We classify these approaches into three categories: conventional methods, those enhanced by AI and Machine Learning, and those based on LLMs. Our findings indicate that LLMs offer significant advantages in contextual reasoning, processing unstructured threat intelligence, and generating actionable mitigation plans. However, challenges such as model explainability, data privacy, system interoperability, and standardization impede their integration into operational environments. In addition to highlighting the potential and practical limitations of LLMs in CTI, this study identifies research gaps and proposes methods to create scalable, secure, and flexible CTI systems that support real-time cyber defense.
Hilalah Alturkistani, Abdul Ghafar Jaafar, S. Chuprat et al.· International journal of res...· 0 citations
Enterprise cybersecurity research draws on a wider range of methods than any single community routinely teaches. Researchers face a selection problem before they face a technical one: a study may simultaneously need a systematic review, a design-science artifact, a controlled detection experiment, an interview study, or an attack-graph model. This paper addresses that problem in two ways. First, it provides a narrative review and synthesis of methodological practices across a verified corpus of 151 works. We organise these practices into eleven methodology families, detailing for each what questions it answers, the strength of its supporting evidence, and its common failure modes. Second, we convert each family into an executable protocol comprising ordered steps, required instruments, evaluation criteria, common validity threats, and a reporting checklist. Every protocol is also visually mapped to make the sequence, decisions, and threats legible at a glance. We also treat contradictions in the literature as evidence. For example, reported rankings of intrusion-detection algorithms are wildly inconsistent across individually careful studies. We argue this pattern is most parsimoniously explained by variations in evaluation design rather than the algorithms themselves, as these studies differ in design dimensions known to shift results by more than the margins separating the algorithms. Ultimately, the evidence supports methodological pluralism disciplined by explicit validity reasoning. We conclude that researchers must match their evaluation design to the decision under study, triangulate technical against organisational evidence, explicitly state the population a result generalises to, and report the conditions under which the result would not hold.
Smart-environment cybersecurity increasingly depends on experimental platforms that can reproduce attacks against buildings, homes, and cities under realistic conditions. However, the literature remains fragmented across testbed design, attack demonstration, and defensive validation. This makes it particularly difficult to judge what kind of security evidence each study actually provides. This review systematically analyses 28 experimentally grounded studies published from 2020 onwards, focusing on how testbed realism, cyber–physical coupling, and evaluation mode shape the strength of the resulting claims. The corpus spans physical, hybrid, emulated, and dataset-driven environments across smart buildings, smart homes, and smart cities. Through our investigation, we discern a clear asymmetry in the field. Detection-oriented studies dominate, especially those based on emulation or public datasets, while live evidence for prevention, response, containment, and recovery is comparatively scarce. Availability and integrity/control attacks are the most frequently exercised, whereas authentication compromise and software exploitation remain rare because they are harder to stage on real hardware. Moreover, an important observation we arrive at is that physical and hardware-in-the-loop platforms support the strongest cyber–physical evidence, but emulated and replayed environments remain valuable for scale and reproducibility. At the same time, public datasets and offline classification results do not by themselves establish operational resilience in a live smart environment. To make these distinctions explicit, we introduce a cross-domain taxonomy of testbed architectures, attack families, and defensive control coverage, and map the evidence strength of reported mitigations using NIST cybersecurity framework-derived operational functions. Last, we identify open challenges, including weak recovery evaluation, limited reuse of reference testbeds, and the need for live, context-aware datasets, outlining promising future directions.
Vyron Kampourakis, Konstantinos E. Kampourakis, Michail Takaronis et al.· Future Internet· 0 citations
Extended Berkeley Packet Filter (eBPF) has emerged as a kernel-level programmable framework enabling dynamic security enforcement in modern operating systems. While eBPF’s potential for cybersecurity applications has attracted significant research attention, existing work remains fragmented across disparate domains, evaluation methodologies, and deployment contexts. This systematic literature review applies PRISMA methodology to identify, categorize, and synthesize peer-reviewed research on eBPF-based cybersecurity mechanisms. Following structured screening of 3735 records from six databases, 54 primary studies published between 2018-2026 were analyzed and organized into a seven-domain taxonomy spanning DDoS mitigation, intrusion detection, Internet-of-Things (IoT) security, container security, microservice protection, networking, and security tools and frameworks. Analysis reveals that eBPF enables low-overhead security enforcement (median 2.4% overhead [1.1–8.6%] of the average CPU usage, ranging from negligible nanosecond-scale costs for infrequently used hooks to higher 10–20% CPU percentages for kernel hot paths) with high detection accuracy (94–99%) across domains, particularly excelling in kernel-level monitoring, real-time packet processing, and cloud-native workload protection. However, significant challenges persist: verifier-imposed constraints limit algorithm complexity, 85.1% (46/54) of studies require low-level programming expertise, kernel version fragmentation hinders portability, and 96.2% (52/54) of research fails to address eBPF’s own security vulnerabilities. The review identifies critical research gaps in multi-tenant isolation, adversarial machine learning (ML) robustness, production validation, and standardized evaluation frameworks. By consolidating fragmented knowledge and highlighting architectural trade-offs between safety and expressiveness, this work provides a foundation for next-generation eBPF security systems and outlines actionable directions for kernel programmability research.
Stamatios Kostopoulos, Panagiotis Tsakonas, Evangelos K. Markakis· International Journal of Inf...· 0 citations
The integration of Artificial Intelligence, Blockchain, and Process Mining into auditing systems promises improvements in automation, transparency, and analytical capability. However, fundamental scientific challenges remain unresolved. This paper presents a systematic literature review of 37 peer-reviewed studies (2023–2026) to identify three structural barriers limiting the development of truly intelligent audit systems: (1) the validity and probative value of digital traces, (2) the faithful reconstruction of real-world processes from imperfect logs, and (3) the detection of complex, rare, and adaptive deviations. Our analysis shows that despite advances in federated learning, deep learning–based anomaly detection, and blockchain audit trails, none fully addresses the foundational data validity problem. The classical "garbage in, garbage out" principle persists across all technological perspectives. Human expertise remains essential for contextual validation, and a persistent trade-off between predictive performance, explainability, and adaptability constrains all current approaches.
Raphael Kpoghomou, David Tchoffa, A. Mhamedi et al.· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
