Skip to content
Book Open access

An Empirical Evaluation of Generative AI in Security Requirements Engineering and Threat Modeling

Jul 2026 · SIGSOFT FSE Companion · 0 citations · 28 references
Computer Science

TL;DR

Empirical evidence is provided that generative AI can effectively support security requirements engineering when embedded within human-centered workflows and organizational governance structures, offering practical insights for adoption in regulated software development contexts.

Abstract

The manual generation of software development artifacts in large organizations—particularly security requirements and threat models—demands substantial effort and is prone to inconsistencies and coverage gaps. While recent advances in generative AI show promise for supporting Requirements Engineering, their adoption in security-critical and regulated environments remains limited due to concerns related to trust, data privacy, and domain specificity. This paper presents an empirical evaluation of an LLM-based AI assistant designed to support security requirements engineering and threat modeling under controlled and auditable conditions. The proposed approach adopts a human-AI hybrid workflow, in which AI-generated artifacts are systematically reviewed and validated by specialists to preserve contextual accuracy and regulatory compliance. Using business documents as input, the assistant generates candidate security requirements and STRIDE-based threat models. The study compares AI-only, manual, and hybrid workflows across 20 real-world projects conducted in a Brazilian public organization. Results show an average reduction of 18.3% in artifact generation time and a 13.6% increase in STRIDE threat coverage, while maintaining 73% semantic precision. Furthermore, the hybrid human-AI approach consistently outperformed the fully manual process in terms of completeness and overall quality. These findings provide empirical evidence that generative AI can effectively support security requirements engineering when embedded within human-centered workflows and organizational governance structures, offering practical insights for adoption in regulated software development contexts.

Read PDF

Similar papers

Book Open access Jul 2026

MAS-SRE: A Multi-Agent System for Security Requirements Engineering

Translating high-level business requirements into standards-grounded security requirements remains a persistent challenge in software engineering. Traditional Security Requirements Engineering (SRE) is often manual, error-prone, and too slow for modern development, creating a translation gap that can leave software vulnerable. This paper presents MAS-SRE, a multi-agent framework that automates the transformation of business requirements into traceable security requirements grounded in OWASP ASVS, NIST SP 800–53, and ISO 27001. MAS-SRE orchestrates 10 specialized agents across 4 workflow stages and combines STRIDE-based threat modeling with Retrieval-Augmented Generation (RAG) to produce standards-aligned outputs. Following the Design Science Research Process, the framework was evaluated on 14 industrial use cases through expert assessment by 15 software engineering practitioners and deterministic coverage and traceability metrics. MAS-SRE achieved 100% verification test coverage, 98.6% threat mapping coverage, 87.6% control mapping coverage, and about 40% lower processing time than sequential execution, while also receiving positive practitioner feedback on usefulness and adoption intent. These results indicate that MAS-SRE is a feasible approach for drafting standards-grounded, traceable security requirements, although comparative evaluation against alternative methods and deeper integration into development workflows remain future work.

Savvas Mantzouranidis, Ricardo Britto · 0 citations
Review Jun 2026

A comprehensive analysis of security risks and productivity impacts of AI-generated code

This systematic literature review examines the dual nature of AI-generated code by analyzing peer-reviewed research to quantify productivity benefits and identify associated risks, and develops a taxonomy categorizing risks into three domains.

Shehnila Zardari, M. Osama, S. Ammar et al. · 0 citations
#artificial intelligence Preprint Jul 2026

From Adoption to Deployment: A Qualitative Study on AI Integration in Software Development Practice

The goal of this study is to understand practitioners'decision making process and security considerations in selecting and integrating AI components through an exploratory semi-structured interview study, and distill the findings into actionable recommendations for AI adopters, model providers, and researchers.

Mahzabin Tamanna, Elizabeth Lin, Sparsha Gowda et al. · 0 citations
Review Jul 2026

Emerging Challenges in Threat Modeling for GenAI-Augmented Systems: A View from the Trenches

This work presents the first results from an exploratory assessment of GenAI-aware threat modeling methods in a Small and Medium Enterprise (SME) setting and reports practitioners' perceptions of the usability and integration of these methods in SME development workflows, including their perceived effort and adoption challenges.

Nicolás E. Díaz Ferreyra, M. Kumar, Nohemí Villarreal et al. · 0 citations
Conference Jun 2026

A Multimodal Large Language Model Approach for Automated Security Requirements Verification

Traditional security auditing and compliance are resource-intensive manual processes. Recent work has applied Large Language Models to automate compliance tasks, but these approaches typically rely on textual inputs, limiting their applicability to real-world audit artifacts. This study proposes a Multimodal Large Language Model (MLLM)-based system for security requirements verification using evidence-grounded analysis of multimodal audit artifacts. The model is constrained to only consider artifacts provided at inference time. Evaluation is conducted through an industrial case study using real audit data, including artifacts, requirements, and ground truth labels. Results show good accuracy and explainability (up to 77%), high consistency, and high recall but lower precision. These findings demonstrate the potential of MLLMs to support automated security requirements verification in practical audit settings.

Ismail Esack Dawoodjee, A. Khurat, Thanapon Noraset · 0 citations