Leveraging Large Language Models for ATT&CK Technique Synthesis: Opportunities and Challenges

Large Language Model-Assisted Threat-Driven Testing System for Enhanced Cybersecurity Readiness

: The rapid evolution of adversarial cyber threats demands proactive, scalable security testing methodologies capable of producing realistic, organization-specific attack scenarios. Conventional approaches, including manual red-teaming, scripted Breach and Attack Simulation (BAS) platforms, and tabletop exercises, are constrained by high expert dependency, limited scenario variability, and an inability to dynamically adapt to an organization’s unique threat profile. This paper proposes and evaluates a Large Language Model (LLM)-Assisted Threat-Driven Testing System that integrates the MITRE Adversarial Tactics, Techniques, and Common Knowledge (MITRE ATT&CK) framework v14, a structured knowledge base of adversarial tactics, techniques, and procedures (TTPs), with GPT-based language models accessed through the OpenAI API, to automate the generation of contextually tailored cyber-attack narratives. The system employs a service-oriented architecture implemented in Python, utilizing Streamlit for the interactive web interface, Pandas for ATT&CK data management, and LangChain as the prompt-orchestration middleware. Evaluation encompassed structured feedback surveys from 30 cybersecurity professionals representing security operations, red-teaming, and incident response roles, together with quantitative analysis using three performance metrics: ATT&CK Technique Coverage (ATC = 85%), False Positive Rate (FPR = 3.2%), and False Negative Rate (FNR = 11%). These results confirm that the system achieves high scenario fidelity, strong ATT&CK alignment, and a generation latency of 2–8 s per scenario. Practically, the framework enables security teams, particularly resource-constrained organizations lacking dedicated red-team capabilities, to conduct high-fidelity threat simulation exercises aligned with current adversarial TTPs, without specialized AI expertise, thereby strengthening organizational cyber-readiness at significantly lower cost than traditional security testing approaches.

Generating Attacks for LLMs with GFlowNets

This study proposes an automated, human-independent, and adaptive approach leveraging GFlowNets to identify LLM vulnerabilities by utilizing one large language model to test another, and introduces a model capable of generating attack inputs in the Turkish language.

Generating Attacks for LLM with GFlowNets

The rapid advancement of Large Language Models (LLMs) has facilitated their ubiquitous integration into various domains, leading to widespread adoption. However, this escalating trend has introduced significant security vulnerabilities, necessitating the identification and mitigation of flaws arising from malicious exploitation. Red teaming assessments, conducted to evaluate model robustness through diverse adversarial inputs, are essential for exposing security risks and implementing countermeasures. Currently, red teaming is performed either manually by experts or automatically using predefined attack datasets. Nevertheless, manual testing remains time-consuming, while existing automated methods suffer from limited creativity due to their inherent dependency on fixed datasets. In this study, we propose an automated, human-independent, and adaptive approach leveraging GFlowNets to identify LLM vulnerabilities by utilizing one large language model to test another. Within this framework, an attacker model is trained against a specified victim model to perform automated red teaming and provide a quantitative robustness score. This research aims to generate more effective adversarial attacks in English compared to existing benchmarks and, as a novel contribution to the literature, introduces a model capable of generating attack inputs in the Turkish language.

A Virtual Lab for Learning AI Security and Adversarial Prompt Engineering

Computing education increasingly focuses on teaching secure coding and developing secure web applications. Now, with the rise of generative AI, we face new challenges, particularly the potential misuse of this technology for identifying and exploiting software vulnerabilities. This paper presents a virtual learning environment that integrates a secure sandbox with access to multiple commercial and open-source LLMs to support experiential learning in AI security. The lab offers scenario-based exercises that cover web, code, and system-level vulnerabilities. Students craft adversarial prompts, test LLM-generated exploits, and analyse models behaviour using established metrics such as attack success rate (ASR), exploit generation accuracy, and refusal rate. Our expert validation revealed distinct model behaviours: GPT-4o achieved the highest ASR (77.5% in web testing), demonstrating consistent exploit generation, while Claude exhibited the highest safety refusal rate (32.56%). By engaging with these observable outcomes, students develop a critical understanding of LLM capabilities, limitations, and ethical risks. We argue that AI-integrated virtual labs are essential for preparing students to work responsibly with emerging AI-assisted security tools.

VectorSec: A Web-Based AI Security Scanner for Systematic Evaluation of LLM Vulnerabilities

A Comparative Analysis of Security Vulnerabilities and Defense Mechanisms in Large Language Models

Large Language Models (LLMs) are now deployed at an unprecedented scale across many critical sectors, rapidly transitioning from experimental AI tools to embedded components of production software systems. This accelerated adoption, often enabled by low-code integrations, has lowered technical barriers while simultaneously expanding the attack surface of modern applications, particularly when deployments occur without sufficient domain-specific security expertise. In many cases, security maturity has not progressed at the same pace as capability expansion, creating systemic exposure across confidentiality, integrity, and availability dimensions. To provide structured clarity amid this rapid growth, this paper presents a comparative and standards-aligned analysis of LLM security risks and defense mechanisms grounded in the OWASP GenAI Top-10 (2025). We systematically examine each vulnerability class, map representative attack patterns to primary mitigation strategies, evaluate their security property impact, and analyze practical limitations and implementation trade-offs. In addition, we introduce a severity-based assessment to prioritize risks according to operational and systemic impact, offering a quantitative perspective on defensive readiness. Our findings indicate that current mitigation strategies are predominantly reactive, concentrated at inference time, and unevenly distributed across the LLM lifecycle. Controls addressing training pipelines, supplychain dependencies, and autonomous system behaviors remain comparatively less mature and less standardized. By integrating vulnerability classification, defense mapping, severity prioritization, and trade-off analysis within a unified framework, this study provides actionable guidance for strengthening secure, resilient, and standards-driven LLM deployment in high-stakes environments.