This paper provides a tutorial and survey of LLM-based agents for automated vulnerability discovery and penetration testing. The rapid advancements in Large Language Models (LLMs) have opened new possibilities for their application in cybersecurity. Unlike traditional tools, LLMs can process natural language, retain context, and perform structured reasoning, enabling automation of complex security workflows. By equipping these models with memory mechanisms and tool-use interfaces, researchers have developed LLM-driven autonomous agents capable of performing specialized tasks such as web application testing, source code auditing, fuzzing, and network security assessment. Multi-agent architectures have emerged in which agents assume specialized roles (e.g., distinct penetration-test phases or exploit types). In addition to surveying existing applications, this work provides a step-by-step walkthrough of integrating agentic AI into penetration testing workflows. The walkthrough explores four phases: (I) mission scoping and prompt engineering for test definition and constraint enforcement, (II) autonomous exploration and tool selection for target interaction, (III) vulnerability hypothesis formation and verification through experiment design and feedback, and (IV) payload generation and refinement to transform validated findings into concrete exploits. The survey synthesizes 155 cited sources (2022–early 2026) using explicit inclusion/exclusion criteria, with coverage extending to reasoning-class models, the Model Context Protocol ecosystem, autonomous bug-bounty agents, and standardized agentic security benchmarks. Despite progress, challenges persist, such as model hallucination, context limitations, operating costs, and ethical concerns. Future directions include advancements in multi-agent collaboration, mechanisms for agent self-improvement, and improved transparency into agent behavior.
Software-Defined Vehicles (SDVs) expand the automotive attack surface across source code, runtime logs, and deployment topologies, while safety constraints forbid autonomous agents from acting without oversight. This paper presents CyberLLM, a multi-agent, LLM-orchestrated framework that autonomously detects vulnerabilities and executes remediations under a formal, runtime safety guard. Detection combines a deterministic layer (regex rules, AST analyzers, and topology graph checks) with an LLM refinement pass, so a high-recall floor is complemented by high-precision reasoning. A decision agent aggregates findings, tags them with a human-centric asset taxonomy, and selects a tiered response, ratcheting its confidence with signed cross-session memory and re-planning feedback. Every action is validated against four contextual security properties and an independent action-alignment oracle before it is allowed to run, and refused actions trigger escalation and re-planning. A symmetric attack pipeline generates and replays exploits so both sides can be exercised on the same scenarios. On an independent, ground-truthed benchmark of nine original automotive ECU modules in C, C++, and Rust seeding 47 layered vulnerabilities plus clean controls, the always-on deterministic layer covers 34\% of the labeled vulnerabilities at perfect precision, and adding the grounded LLM refinement and completeness passes roughly doubles coverage to about 70\% (F1 $0.83$) while producing zero false positives on the clean controls. The results indicate that LLM agents can perform useful autonomous cyber-defense when wrapped in a deterministic, auditable safety envelope.
Nenad Petrovic, Oussama Jeddou, Feres Ben Fraj et al.· 0 citations
A multi-agent GenAI architecture is introduced to support the automation of ethical cloud security, solving the problem of scalability or adaptability, and compliance in dynamic cloud environments. This framework combines dedicated generative agents such as policy analysts, threat detectors, remediation organizers, and auditor agents which interact via common knowledge graph and can be explained by a decision log. The agents utilize context-based prompt generation, generation constraints, provenance management, and generation to generate security policies, anomaly detection, provide automated mitigation, and maintain human-in-the-loop control. Some of the ethical protections are bias audits, privacy-sensitive learning, least-privilege enforcement, and policy verifiability, to warrant the correctness of the decisions taken in compliance with regulatory or organizational limits. Testing with representative cloud work lines shows that there is shorter energy on discerning and correcting occurrences, elevated coverage of controls and signs when the automated actions are traced. The framework enables adjustable levels of trust and escalation measures to accommodate the autonomy versus governance. The method fosters usage of GenAI to deal with cloud security by offering modular agents, verifiable ethics controls, metrics-based assessment, hence fostering responsible automation that is also transparent, auditable, and considers changing threats.
Laxminarayana Thirupathi, Sharanya Gattu, T. Wable et al.· 2026 International Conferenc...· 0 citations
LLM-based agents are rapidly moving from research prototypes into the core business processes of organizations, but these agents pose deployment risks to security, compliance, and functionality. In this article, we argue that risk-free deployment must be grounded in the agent's trajectory: the recorded sequence of reasoning steps, tool invocations, and environmental observations. Trajectories are available for any agent, and many failures are visible only in the trajectory. To make agents deployable and sustainable, we advocate agent testing and debugging as a systematic research direction for detecting and mitigating these risks. This article begins with the challenges of testing agents, including the oracle problem, non-determinism, trajectory validation, and the absence of adequacy metrics. We then turn to debugging agents, from automated failure attribution to repair and self-evolution. We distill these directions into a practical deployment-readiness checklist covering the full deployment lifecycle. Finally, we identify open problems, i.e., formal adequacy metrics, root-cause attribution over long-horizon trajectories, and the reliability of self-evolving agents, that the community must address to enable trustworthy agent deployment.
Software vulnerabilities have become a recognized national-security risk, yet the volume of code and the sophistication of threats now far outpace what manual security review and the limited supply of expert security engineers can sustain. A new class of systems has emerged in response: agentic systems for software security detection, which couple large language models with planning, memory, and external tools so that they can autonomously analyze codebases, reason about program behavior, and identify, triage, and help re-mediate vulnerabilities. The field has grown rapidly but unevenly, and its terminology, capabilities, and evaluation practices remain fragmented. This paper organizes the area into a structured taxonomy along five axes: the de-tection capability targeted, the analysis paradigm employed, the agent archi-tecture, the degree of autonomy, and the evaluation methodology. We popu-late the taxonomy with representative systems, including the cyber reasoning systems demonstrated at the DARPA AI Cyber Challenge, and we use it to compare designs, surface recurring patterns, and expose gaps. We find that the strongest results combine learned reasoning with classical program analy-sis and tool use rather than relying on either alone, and that repository-scale detection, trustworthy triage, and reproducible evaluation remain the principal open challenges. The taxonomy is intended as a shared vocabulary and a roadmap for building the next generation of autonomous software-security systems.
Andi Xia· Poster Volume 0008 The 2026...· 0 citations
The rapid development of large language model (LLM) agents has enabled their broad adoption across diverse real-world tasks. To standardize interactions between LLM agents and external environments, Model Context Protocol (MCP) tools have emerged as a de facto standard and have been widely integrated into these systems. However, the use of MCP tools also introduces new safety risks, as LLM agents can be induced to perform malicious or unauthorized actions. Although prior work has proposed defenses for securing tool use in LLM agents, most methods rely on static analysis, i.e., inspecting prompts and generated outputs, which limits the defense effectiveness and robustness. To address these limitations, we propose MTGuard, a hybrid analysis-based defense framework designed to safeguard the use of MCP tools in LLM agents by leveraging lifecycle-aware static-dynamic co-analysis. Extensive evaluation demonstrates that MTGuard effectively mitigates multiple categories of harmful tool use across different LLM agents while maintaining performance on benign user tasks.
Ping He, Yuexiang Xie, Yaliang Li et al.· 0 citations
Large Language Models (LLMs) have undergone a shift from stateless conversational interfaces to autonomous agents capable of multi-step planning, tool invocation, code execution, and maintaining persistent memory. When these agents operate with real-world privileges---calling APIs, modifying files, and querying databases---a compromised reasoning step can trigger unauthorized data access, irreversible state changes, or cascading failures, yet the security research community has not kept pace. To quantify the state of the field, we conducted a systematic literature review under PRISMA 2020 guidelines across six databases, screening 743 records and retaining 85 papers (2023--2025) on agentic LLM security. Attack research outpaces defense work by 3.9:1. Perception-layer vulnerabilities (prompt injection, jailbreaking, adversarial perturbations) dominate, accounting for 66\% of papers, while action-layer vulnerabilities (tool misuse, code injection, sandbox escape) appear in only 4.7\%, misaligned with real-world risk. Code execution security accounts for 3.5\%, and tool-augmented agents 12\%. We contribute a four-layer taxonomy mapping 13 vulnerability types across perception, brain, action, and interaction layers, and identify seven open problems centered on containment. Agentic LLM insecurity stems from architectural coupling, where weak isolation allows vulnerabilities to propagate across layers.
Md. Jafrin Hossain, Mohammad Arif Hossain, Nirwan Ansari· 0 citations