This survey provides a comprehensive analysis of adversarial prompting strategies, ranging from input manipulation techniques to semantic and structural distortions, and explores defense strategies across preprocessing, model-level, postprocessing, and hybrid strategies, highlighting recent advances and their limitations.
Abstract
The rapid integration of Large Language Models (LLMs) into critical domains has amplified concerns about their robustness, particularly against adversarial prompts that exploit subtle vulnerabilities. These crafted inputs can cause models to generate biased, misleading, or harmful outputs, posing serious ethical, societal, and operational risks. In this survey, we provide a comprehensive analysis of adversarial prompting strategies, ranging from input manipulation techniques to semantic and structural distortions. We systematically review benchmark datasets and evaluation metrics developed to assess model resilience under adversarial conditions. Furthermore, we explore defense strategies across preprocessing, model-level, postprocessing, and hybrid strategies, highlighting recent advances and their limitations. Special attention is given to emerging challenges, such as multimodal vulnerabilities, the balance between safety and expressiveness, and ethical considerations in adversarial defense design. Finally, we outline future research directions that aim to build LLMs that are not only more robust and trustworthy but also adaptable to evolving adversarial tactics. By synthesizing the current landscape, this work serves as a foundation for advancing the secure deployment of LLMs in real-world applications.
A new perspective of information theory is introduced to investigate LVLMs’ transferable characteristics by exploring the relative dependence between outputs of the LVLM model and input adversarial samples and formulate the complicated calculation of information gain as an estimation problem and incorporate such informative constraints into the adversarial learning process.
Xiaowen Cai, Daizong Liu, Xiaoye Qu et al.· Neural Information Processin...· 7 citations
This study systematically examines the vulnerabilities of open and closed-weight LVLMs, including LLaVA, InstructBLIP, Fuyu, Qwen, DeepSeek, Gemini, GPT, and Grok, using adversarial prompting strategies informed by social theories to simulate real-world social manipulation tactics.
Abdulkadir Erol, Trilok Padhi, Agnik Saha et al.· ACM Transactions on Intellig...· 0 citations
This study proposes an automated, human-independent, and adaptive approach leveraging GFlowNets to identify LLM vulnerabilities by utilizing one large language model to test another, and introduces a model capable of generating attack inputs in the Turkish language.
Berkay Ozcam, Irem Onen, M. Amasyalı et al.· 0 citations
This paper proposes a novel LVLM attack method, called BadPhase with further backdoor designs, to implant adversarial phase as triggers into any image inputs via data poisoning so as to control the LVLMs’ predictions and finds that LVLMs are sensitive to the phase-aware image structure.
Daizong Liu, Junhao Dong, Xiang Fang et al.· 0 citations
The rapid advancement of Large Language Models (LLMs) has facilitated their ubiquitous integration into various domains, leading to widespread adoption. However, this escalating trend has introduced significant security vulnerabilities, necessitating the identification and mitigation of flaws arising from malicious exploitation. Red teaming assessments, conducted to evaluate model robustness through diverse adversarial inputs, are essential for exposing security risks and implementing countermeasures. Currently, red teaming is performed either manually by experts or automatically using predefined attack datasets. Nevertheless, manual testing remains time-consuming, while existing automated methods suffer from limited creativity due to their inherent dependency on fixed datasets. In this study, we propose an automated, human-independent, and adaptive approach leveraging GFlowNets to identify LLM vulnerabilities by utilizing one large language model to test another. Within this framework, an attacker model is trained against a specified victim model to perform automated red teaming and provide a quantitative robustness score. This research aims to generate more effective adversarial attacks in English compared to existing benchmarks and, as a novel contribution to the literature, introduces a model capable of generating attack inputs in the Turkish language.
Berkay Özçam, İrem Önen, E. I. Tatli et al.· Signal Processing and Commun...· 0 citations
As large language models are increasingly deployed in real-world systems, safety failures can still lead to harmful outputs and dangerous misuse. We argue that the essence of safety is adversarial: many failures arise not from natural inputs alone, but from strategic attempts to evade model policies and safeguards. However, existing general-purpose model development largely overlook this adversarial nature, and often remain insufficient for realistic safety scenarios involving planning, tool use, and multi-step reasoning, causing measured safety performance to overestimate real deployment robustness. To address this gap, we present Yuvion LLM, a large language model built for adversarially robust content safety and broader AI safety. Yuvion LLM treats adversarial robustness and agentic capability as first-class objectives. Its pipeline combines adversarially aware data construction, knowledge-enhanced continued pretraining, and policy-grounded multi-task safety post-training, including risk-aware supervised fine-tuning and reinforcement learning-based policy optimization, together with safety-aware agentic reinforcement learning for tool use and multi-step reasoning in complex safety scenarios. We further introduce the Yuvion LLM RiskEval (YLRE), a collection of 93 benchmarks across four evaluation categories, covering diverse open and internal evaluations with a focus on safety, adversarial robustness, and real-world capability requirements. Across these evaluations, Yuvion LLM demonstrates clear advantages on safety-focused benchmarks and particularly strong robustness under adversarial conditions, while maintaining solid overall capability. Notably, Yuvion-8B outperforms most state-of-the-art baselines, including substantially larger models such as GPT-5.4 and Qwen3-MAX, on several safety tasks.
Ting Ma, Xiufeng Huang, Benlei Cui et al.· 0 citations