Multi-dimensional Vulnerability Analysis of LM Safety Guards via Role-Goal-Format Decomposition

Large Language Model Vulnerabilities

Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings

Reflex-Guard is introduced, a lightweight guardrail that runs locally that uses jailbreak-aware preprocessing, compact sentence-transformer embeddings, and seven fast binary classifiers that enable high-accuracy prompt safety filtering with much lower latency than existing solutions.

TSA: A Two-Stage Jailbreak Attack Exploiting Logical Consistency of Large Language Models

Large language models (LLMs) are widely deployed in high-stakes decision-making tasks, raising growing security concerns. Jailbreak attacks, a major threat to LLMs, have evolved from superficial semantic evasion to exploiting inherent model properties. LLMs exhibit a tendency toward logical consistency: once a premise is accepted, models tend to follow its reasoning chain, which may lead them to generate harmful content even if the final output violates safety rules. This tendency may present a potential vulnerability that could be exploited for jailbreak attacks. To exploit this vulnerability, this paper proposes TSA (Two-Stage Jailbreak Attack), a lightweight two-stage jailbreak framework. The method consists of two core steps: first, logic presetting, which guides the model to generate a structured analysis report of harmful behavior and establishes a compliant logical premise; second, intent enhancement, which extracts execution paths from the generated analysis content and prompts the model to autonomously produce harmful outputs. Evaluations on nine mainstream LLMs show that TSA achieves an average attack success rate (ASR) of 84.83% on MiniAdvBench with only 3.61 Queries Per Successful Jailbreak (QPS) and an average ASR of 51.44% on MiniHarmBench, performing favorably compared with the evaluated baseline methods under our experimental settings. The findings suggest that such logical-consistency-based vulnerability may exist among the tested mainstream LLMs, highlighting the necessity to optimize safety alignment for defending against this category of reasoning-driven jailbreak attacks.

The Anatomy of a Prompt Injection: A Component Model for Structured Analysis

Four years after prompt injection was first identified in 2022, attacks are still predominantly documented as verbatim strings rather than structured exploits, despite advancing agent capabilities and threat actors embedding injections to subvert AI-assisted security analysis. This paper formalizes the structure of prompt-injection artifacts, enabling defenders, red teamers, and cyber threat intelligence (CTI) teams to label, compare, and mutate attacks without relying on fragile string matching. Because large language models compile varied natural-language realizations into identical executable actions, labeling must track attacker intent (tool targets, sinks, and effects) rather than surface wording. We propose a seven-component model (carrier, delivery vector, concealment, context-break, privilege escalation, payload, and return channel) consisting of five artifact fields and two environment fields. This framework unifies roles partially addressed by HOUYI's payload decomposition, the Promptware Kill Chain, and campaign taxonomies, while framing minimal jailbreak frameworks like ReNeLLM as projections onto a restricted subspace. We provide clear labeling rules, a logical analysis record mapping directly to industry CTI schemas, worked examples including EchoLeak (CVE-2025-32711) and an in-the-wild malware AI-evasion sample, and an illustrative agentic flowchart.

OmniVul: A Holistic, Multi-Turn Conversational Benchmark for LLM-Based Vulnerability Assessment

With more than 20,000 Common Vulnerabilities and Exposures (CVEs) reported annually, software vulnerabilities represent a critical cybersecurity challenge. This volume has intensified the demand for automated detection and analysis, motivating the integration of large language models (LLMs) for such tasks. However, existing vulnerability benchmarks are not suitable for evaluating LLMs' capabilities in vulnerability assessment, as most of them 1) rely on narrow data sources, 2) lack deep context, and 3) focus on single-turn Q&A rather than realistic, multi-stage analyst workflows. To address this gap, we introduce OmniVul, a comprehensive multi-turn benchmark for LLM-based vulnerability assessment. OmniVul comprises 2,000 CVEs with question–answer pairs spanning 23 attributes, including detection, code localization, root cause analysis, and patch suggestion. We employ an automated workflow to aggregate multi-source data via Retrieval-Augmented Generation (RAG), ensuring quality through LLM-as-a-Judge filtering and conformal prediction calibrated by human expert annotations. An evaluation of five state-of-the-art LLMs on OmniVul reveals distinct performance gaps, with top-1 accuracy remaining below 50% on average for vulnerable code detection and CVE identification. Our evaluation also demonstrates that current models lack critical reasoning capabilities for reliable vulnerability assessment. These results highlight the importance of OmniVul for advancing research in evaluating and fine-tuning LLMs for vulnerability assessment.

Red Teaming LLMs via Linguistic-Aware Fuzzing

Safety alignment aims to prevent Large Language Models (LLMs) from producing harmful content. However, safety alignment remains vulnerable to malicious instructions. Red teaming is a critical methodology for identifying such vulnerabilities in LLMs. Existing approaches often rely on jailbreak templates or rule-based transformation, limiting the diversity of generated tests and the continued testing capability of these test approaches. To address these limitations, we propose Lingfuzz, a linguistic-aware fuzzing framework for continuing red teaming LLMs. The key idea of this framework is to mutate the existing malicious instructions at the lexical and syntactic levels, while keeping the malicious intentions of the instructions. Such mutations enable generating diverse malicious instructions due to the unlimited space of lexical and syntactic choices, while having a continued testing capability by iteratively mutating the mutants. We evaluated Lingfuzz on five aligned commercial LLMs and one open-source LLM in black-box testing. The results show that Lingfuzz triggers safety alignment vulnerabilities in 71.0% of the cases in JailbreakBench benchmark, higher than the second-highest baseline of 63.8%. Lingfuzz also possesses outstanding multilingual testing capabilities that far exceed other baselines with approximately 24% ESR advantage testing in Jailbench benchmark. The malicious instructions generated by Lingfuzz have almost three times higher diversity than previous work according to the self-BLEU metric. Lingfuzz also demonstrates strong continued testing capability by showing five times less sensitivity to the LLM evolution than other approaches.