Jul 2026· IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies· pp. 143-147· 0 citations· 17 references
Abstract
The threat posed by adversarial prompts to large language models is becoming harder to ignore. Problems including prompt injection, jailbreaking, phishing, and Unicode-based attacks are now widespread. Most existing solutions protect against only one threat type, operate in English only, and provide no explanation for their decisions. We present SemGuard, a multilingual security gateway using Triple-Anchor Semantic Threat Modeling, which simultaneously evaluates each input against three semantic reference sets: attack, safe, and destructive. SemGuard detects four threat types concurrently in Arabic, Arabizi, and English. We expand the original Arabic Security Dataset from 319 to 807 validated examples across seven threat categories, using three independent LLM judges (GPT-4o, Grok-4, Llama 3.3 70B) achieving Fleiss' $\kappa=0.839$. After retraining on the expanded dataset, SemGuard achieves a mean F1-score of 0.989 and recall of 0.991, representing a 13.7% improvement over the original implementation. Analysis of 527 rejected examples reveals quantitative evidence of threat-category ambiguity, with impersonation exhibiting a 98.2% inter-judge disagreement rate, validating the necessity of the Triple-Anchor framework. This work also presents the first Arabic LLM security dataset with a formal LLM-as-Judge annotation protocol.
The increasing deployment of Large Language Models (LLMs) in critical infrastructure has introduced a class of security risks that remain insufficiently characterised and poorly tooled in practice. Attack vectors including prompt injection, jailbreaking, code execution facilitation, covert data exfiltration, and training data poisoning present quantifiable threats to systems that rely on LLM outputs, yet no lightweight, provider-agnostic tool exists to measure these risks systematically. This paper presents VectorSec, an open-source, web-based LLM security scanner built with Python and Dash. The tool executes a structured test suite of 320 adversarial prompts spanning 16 vulnerability categories aligned with the OWASP Top 10 for LLMs. Each response is scored through a four-stage pipeline combining pattern matching, sentiment analysis, semantic similarity, and secondary LLM verification. Evaluation across three open-source models demonstrates that pattern-matching-only filtering misses approximately 15% of High and Critical findings that semantic verification correctly identifies, motivating the multi-layer design. VectorSec provides interactive dashboard reporting, real-time progress tracking, and audit-ready PDF and CSV exports, lowering the barrier to structured LLM security assessment in operational settings.
M. Yamin· International Conference on...· 0 citations
This survey offers a comprehensive overview of the main data security risks facing LLMs and reviews current defense strategies, including adversarial training, data cleaning, output guardrails, Reinforcement Learning from Human Feedback, data augmentation, and Retrieval-Augmented Generation (RAG)/agent defenses.
Kang Chen, Xiuze Zhou, Yuanhui Yu et al.· Journal of King Saud Univers...· 0 citations
A benchmark for this vulnerability in LLM-based resume screening is introduced: 463 job-candidate pairs drawn from a 14-domain corpus, with the evaluated sample covering 13 domains, attacked through a taxonomy of four attack types and four injection positions.
Hong-Lin Mu, Jinghao Liu, Kaiyang Wan et al.· International Journal of Mac...· 3 citations· ⚡1
As the adoption of large language models (LLMs) grows in Arabic-speaking regions, ensuring their safety and cultural alignment is increasingly critical. However, Arabic LLM safety remains underexplored, especially in adversarial evaluation settings. We introduce the Arabic Safety Index (ASAS), the first fully human-curated Arabic benchmark for redteaming LLMs. ASAS contains 801 prompts spanning 8 safety categories and 8 attack strategies, with ideal responses in Modern Standard Arabic (MSA). We conduct a redteaming evaluation across seven leading models with Arabic capabilities, including GPT-4o, Claude 3.7 Sonnet, and regional models such as ALLaM and FANAR. Human annotators rate responses using a structured 4-point safety scale, revealing that most models fail to defend against 50% of unsafe prompts. Our findings highlight major safety gaps in high-harm categories such as weapons and illicit substances, with direct and obfuscation-based attacks proving most effective. The results also show that language alignment does not readily transfer across languages, and that automated safety judges (e.g., GPT-4o) perform poorly compared to human annotators. ASAS provides a culturally grounded benchmark and redteaming protocol to drive progress in Arabic LLM safety.
Fidaa Abed, Haidar Khan, M Saiful Bari et al.· 0 citations
Large language models (LLMs) are vulnerable to jailbreak attacks that bypass safety alignment through carefully crafted prompts. Many existing defenses require access to model weights or internals, making them difficult to apply to black-box deployments. We propose AlcaTRAz (Anchored Tree-Rule defense Against jailbreaks), a prompt-level defense based on rule trees that operates exclusively on the input text and requires no modification or retraining of the target model. The method automatically learns a transferable transformation rule that inserts controlled character-level perturbations at selected positions, thereby disrupting structural regularities exploited by jailbreak attacks while largely preserving the model's utility on benign queries. We evaluate the proposed method across 33 open-weight models, 22 jailbreak attack types, and a benchmark of short, single-turn benign questions, comparing against three representative prompt-level baselines (Llama Guard, RA-LLM, Goal Prioritization). Among the compared defenses, AlcaTRAz achieves the best composite security and functionality score in 73.4 % of model-attack combinations and shifts the aggregate score from a modal value of 10 (maximal-severity response to the malicious request) in the undefended setting to a modal value of 2 (near-refusal) after defense, while keeping the mean benign score within 0.27 points of the undefended baseline (8.35 vs. 8.62 on a 0-10 scale). AlcaTRAz substantially reduces but does not eliminate jailbreak success: a high-severity tail remains, and we do not consider adaptive attackers, so we position it as one layer within a defense-in-depth strategy rather than a standalone guarantee.
J. Res, Petr Kaska, Martin Perešíni et al.· 0 citations
This paper formalizes the structure of prompt-injection artifacts, enabling defenders, red teamers, and cyber threat intelligence (CTI) teams to label, compare, and mutate attacks without relying on fragile string matching.
Jeremy McHugh· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
