Jul 2026· IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies· pp. 1-6· 0 citations· 17 references
Abstract
With the increasing usage of AI-generated code in software development workflows, new security challenges and concerns arise. This paper analyzes five LLMs: ChatGPT, Claude, Gemini, DeepSeek, and Grok in three phases of security assessments against web vulnerabilities listed by the OWASP Top 10. Phase 1 (December 2025) evaluated 13 prompts in XSS, Authentication, API Security, Hardcoded Secrets, and React Server Component CVE. Phase 2 (March 2026) evaluated 11 prompts using newly disclosed CVEs, complex authentication, and API security, with updated models. Phase 3 (May 2026) revisited phase one prompts against current models. Across all 185 samples, 49 out of the 120 Phase 1 and 2 samples (40.8%) were identified as vulnerable, 65 (54.2%) as secure and 6 (5.0%) partially vulnerable. The CVE-based prompts recorded the highest percentage of vulnerability (76%). No failures were observed in the authentication prompts with simple patterns, while complex access control resulted in 45% failures. Phase 3 (65 samples) re-ran Phase 1 prompts on current models; no progress was observed; the partial result rate went up from 7.7% to 27.7%. The results revealed hedged responses from the updated models. All results were cross-validated using Semgrep static analysis.
The risk is not tied to any particular model but rather to the task category, and organizations should ask not which tool to trust, but instead whether LLM-generated automation code should be deployed without review.
Shanna M. Kahn, John D. Hastings· arXiv.org· 0 citations
The increasing deployment of Large Language Models (LLMs) in critical infrastructure has introduced a class of security risks that remain insufficiently characterised and poorly tooled in practice. Attack vectors including prompt injection, jailbreaking, code execution facilitation, covert data exfiltration, and training data poisoning present quantifiable threats to systems that rely on LLM outputs, yet no lightweight, provider-agnostic tool exists to measure these risks systematically. This paper presents VectorSec, an open-source, web-based LLM security scanner built with Python and Dash. The tool executes a structured test suite of 320 adversarial prompts spanning 16 vulnerability categories aligned with the OWASP Top 10 for LLMs. Each response is scored through a four-stage pipeline combining pattern matching, sentiment analysis, semantic similarity, and secondary LLM verification. Evaluation across three open-source models demonstrates that pattern-matching-only filtering misses approximately 15% of High and Critical findings that semantic verification correctly identifies, motivating the multi-layer design. VectorSec provides interactive dashboard reporting, real-time progress tracking, and audit-ready PDF and CSV exports, lowering the barrier to structured LLM security assessment in operational settings.
M. Yamin· International Conference on...· 0 citations
An automated security evaluation pipeline that generates Python code from LLMSecEval prompts, scans for vulnerabilities using CodeQL and Bandit in parallel with an independent Code Validator LLM, enriches the Code Validator findings with MITRE ATT&CK techniques, CWE Observed Examples, and Python best practice guidelines, generates fixes via the Code Generation LLM, and re-scans with CodeQL and Bandit to verify outcomes is presented.
An empirical benchmark evaluating whether LLMs and SLMs can generate security-compliant AWS Terraform configurations suggests that prompt design is a critical factor, highlighting the need for a proper pipeline for developing and validating LLM-assisted secure IaC generation.
Francis Luis Santos Vargas, R. Mansilha, Diego Kreutz· Anais do I Simpósio de Infra...· 0 citations
Investigating the security vulnerabilities present in AI-generated JWT authentication code for Java Spring Boot Representational State Transfer Application Programming Interfaces (REST API) reinforces that AI-generated JSON Web Token (JWT) authentication code requires dedicated security review.
Hoang Long Nguyen, Mezid Hmudda, Benjamin Powley· 0 citations
Large language models (LLMs) can rapidly generate web user interface (UI) code for intelligent multimedia applications, but the resulting code may contain avoidable client-side security weaknesses. This paper presents a security governance framework that combines pre-generation rule embedding, ESLint-based static analysis, structured feedback, and LLM-based repair. The framework was evaluated through a controlled pilot ablation study involving 10 multimedia UI requirements, five experimental groups, and three repeated runs, producing 150 requests. Across all groups, 132 outputs passed structural and task-specific validity checks, while 18 were recorded as generation failures rather than being treated as secure code. For the Full Framework, the matched-pair end-to-end static finding reduction rate was 95.16% (SD = 4.76 percentage points), the valid output rate was 90.00%, and the static-secure valid output rate was $\mathbf{7 6. 6 7 \%}(\mathbf{S D}=$ 11.55 percentage points). Holm-adjusted paired Wilcoxon tests showed statistically significant differences, with lower observed final finding counts for the Full Framework than for Vanilla, Prompt Only, and the variant without structured static feedback. Repair-enabled configurations increased the average runtime from approximately 51 s to 79 s. These results concern static candidate findings and output validity; they do not replace server-side controls or exhaustive dynamic security testing.
Shu-Ning Ping, Guangchi Liu· 2026 International Conferenc...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
