Jul 2026· 2026 IEEE 9th International Conference on Big Data and Artificial Intelligence (BDAI)· pp. 181-186· 0 citations· 10 references
Abstract
With the rapid development of large language models (LLMs), LLM-based agents are evolving from single-turn text generators into autonomous decision-making systems capable of task planning, tool invocation, memory retrieval, and environmental interaction. However, the openness of agent environments and the nondeterminism of model-driven decision logic introduce security and reliability risks that are difficult to capture with conventional software testing. Fuzzing, as an automated and scalable vulnerability discovery technique, provides a promising approach for assessing the robustness of LLM agent systems. This paper presents a systematic survey of fuzzing techniques for LLM agents. First, we analyze the distinctive architecture of LLM agents and summarize three representative threat categories: indirect prompt injection, planning hallucination, and tool abuse. Second, we review existing fuzzing methods from three perspectives: test-case generation and mutation, environment simulation and interaction monitoring, and test-oracle construction. Particular attention is paid to LLM-enhanced fuzzers that exploit the semantic understanding and generation capabilities of LLMs to improve mutation quality and attack effectiveness. Finally, we connect the main technical challenges with an actionable research roadmap, including semantic coverage, cost-aware execution, reproducible oracles, multi-agent fuzzing, gray-box guidance, and automated repair.
A four-layer taxonomy mapping 13 vulnerability types across perception, brain, action, and interaction layers is contributed, and seven open problems centered on containment are identified.
Md Jafrin Hossain, Mohammad Arif Hossain, Nirwan Ansari· 0 citations
Large language model (LLM) agents extend generative models with planning, memory, and external tool access, but
this capability creates a security path in which untrusted content can alter instructions, hijack an agent's operational goal, and
trigger harmful tool actions. This paper proposes TRACER-AI, a four-layer explainable defense-in-depth framework that
combines (i) semantic prompt-injection detection, (ii) continuous goal-integrity monitoring, (iii) contextual tool-risk control, and
(iv) structured explainable security decisions. The framework is designed around the attack progression prompt injection -> goal
hijacking -> tool misuse rather than treating prompt filtering as the only enforcement boundary. A dynamic risk score fuses
prompt-injection probability, goal deviation, tool risk, and contextual anomaly before action execution. A controlled proof-ofconcept evaluation was conducted on a 3,500-case synthetic adversarial testbed containing benign interactions and five attack
families: direct prompt injection, indirect prompt injection, goal hijacking, tool misuse, and chained attacks. The held-out test set
comprised 1,050 cases with previously unseen attack wording and benign security-text decoys. The standalone prompt detector
achieved 0.679 accuracy, 0.575 F1-score, and 0.760 ROC-AUC, illustrating the weakness of relying on prompt detection alone
under distribution shift. In contrast, the full TRACER-AI configuration achieved a 96.4% attack detection rate, reduced attack
success rate to 3.6%, preserved 99.0% benign task success, and limited false positives to 1.0% in the controlled testbed. The
results support the central hypothesis that agent security benefits from multiple independent checkpoints spanning instruction
intake, goal continuity, and execution-time tool authorization. The study also maps the framework to contemporary agentic-AI
security guidance and benchmark research, and provides a reproducible experimental protocol for subsequent validation on
AgentDojo, InjecAgent, AgentDyn, and domain-specific agent benchmarks.
Pallavi Singh, Khushboo Gupta, Pratibha Singh· International Journal for Re...· 0 citations
This paper constructs a large-scale dataset of agent applications, tools, and tests, and manually label 2,572 test methods from 240 modules, and derives a taxonomy of 23 testing patterns across test fixtures, data, objectives, and assertions, and characterize tests by level.
Rangeet Pan, Tyler Stennett, D. Sankar et al.· 1 citation
This report synthesizes peer-reviewed and archival research published between 2019 and 2024 on the use of large language models and multi-agent architectures for automated vulnerability repair, with a specific focus on how these techniques apply, or fail to apply, to legacy software systems. The review traces a clear methodological progression across this five-year window: from template and heuristic program repair, through neural and transfer-learning models, to fine-tuned transformer-based repair, zero-shot and conversational prompting of general-purpose LLMs, and finally to multi-agent pipelines in which specialized agents divide the work of detecting, localizing, patching, and validating a vulnerability. Two figures accompany the discussion. The first depicts a generalized multi-agent repair pipeline synthesized from the architectures described across the reviewed literature, showing how detection, localization, patch generation, and validation agents typically interact, including the validation feedback loop that distinguishes conversational and agentic systems from earlier single-shot approaches. The second traces the field's methodological evolution chronologically, from 2019's template-based baselines through 2024's agentic systems. A comparative table consolidates fifteen of the most relevant studies discussed in the body of the report, listing the approach, target language, and publication venue for each. The report concludes that while multi-agent LLM pipelines show real promise for legacy code specifically, since their capacity for autonomous context reconstruction and iterative, feedback-driven refinement directly addresses the sparse documentation and thin test coverage that define legacy environments, the literature to date has been validated almost exclusively on actively maintained, comparatively well-organized open-source repositories. The gap between these benchmarks and the scale and disorganization of real enterprise legacy systems has not yet been closed empirically, and closing it, together with building trust and explainability mechanisms suitable for security-critical maintenance work, is identified as the field's most consequential open problem.
Harsh Verma· International Journal of Sci...· 0 citations
This paper proposes SafeKeep, an inference-time safeguard that decouples safety judgment from tool execution: it assesses requests using flattened textual tool specifications while retaining the original schema-formatted specifications for execution.
Multi-agent systems built on large language models (LLMs) are increasingly deployed for complex tasks requiring autonomous planning, tool use, and inter-agent coordination. However, the non-deterministic nature of LLM outputs and the emergent behavior arising from agent interactions render traditional test oracles ineffective, creating a critical gap in quality assurance for agentic AI. This work introduces MORPHAGENT, a framework designed to address the oracle problem in multi-agent LLM systems through trace-based behavioral analysis. Our contributions are threefold: (1) goal-preservation relations that verify consistent goal achievement under input perturbations, (2) coordination-consistency relations that validate inter-agent delegation and communication patterns under agent substitution and reordering, and (3) tool-use integrity relations that ensure semantic equivalence of tool invocation sequences under prompt paraphrasing. MorphAgent instruments agent execution to capture structured traces comprising planning steps, tool calls, message exchanges, and final outputs, then systematically applies metamorphic transformations and checks behavioral invariants without requiring ground-truth oracles. We evaluate the framework on four multi-agent benchmarks spanning code generation, research synthesis, customer service, and data analysis tasks, encompassing 2,840 source-followup execution pairs across three LLM backends. Results show that MORPHAGENT detects 82.0% of seeded behavioral faults, including 90.3% of coordination failures and 81.7% of goal-deviation faults, while maintaining a false positive rate of 6.1%. The framework uncovers 14 previously unreported behavioral anomalies in established multi-agent frameworks, demonstrating its practical utility for assuring agentic AI reliability. These results suggest that trace-based metamorphic testing can serve as a practical foundation for reliable validation of emerging agentic AI systems.
Gopalakrishnan Marimuthu· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
