Large Language Model-Assisted Threat-Driven Testing System for Enhanced Cybersecurity Readiness
TL;DR
The proposed Large Language Model-Assisted Threat-Driven Testing System enables security teams, particularly resource-constrained organizations lacking dedicated red-team capabilities, to conduct high-fidelity threat simulation exercises aligned with current adversarial TTPs, without specialized AI expertise, thereby strengthening organizational cyber-readiness at significantly lower cost than traditional security testing approaches.
Abstract
: The rapid evolution of adversarial cyber threats demands proactive, scalable security testing methodologies capable of producing realistic, organization-specific attack scenarios. Conventional approaches, including manual red-teaming, scripted Breach and Attack Simulation (BAS) platforms, and tabletop exercises, are constrained by high expert dependency, limited scenario variability, and an inability to dynamically adapt to an organization’s unique threat profile. This paper proposes and evaluates a Large Language Model (LLM)-Assisted Threat-Driven Testing System that integrates the MITRE Adversarial Tactics, Techniques, and Common Knowledge (MITRE ATT&CK) framework v14, a structured knowledge base of adversarial tactics, techniques, and procedures (TTPs), with GPT-based language models accessed through the OpenAI API, to automate the generation of contextually tailored cyber-attack narratives. The system employs a service-oriented architecture implemented in Python, utilizing Streamlit for the interactive web interface, Pandas for ATT&CK data management, and LangChain as the prompt-orchestration middleware. Evaluation encompassed structured feedback surveys from 30 cybersecurity professionals representing security operations, red-teaming, and incident response roles, together with quantitative analysis using three performance metrics: ATT&CK Technique Coverage (ATC = 85%), False Positive Rate (FPR = 3.2%), and False Negative Rate (FNR = 11%). These results confirm that the system achieves high scenario fidelity, strong ATT&CK alignment, and a generation latency of 2–8 s per scenario. Practically, the framework enables security teams, particularly resource-constrained organizations lacking dedicated red-team capabilities, to conduct high-fidelity threat simulation exercises aligned with current adversarial TTPs, without specialized AI expertise, thereby strengthening organizational cyber-readiness at significantly lower cost than traditional security testing approaches.