Nov 2026· Proceedings of the 34th ACM International Conference on Multimedia· pp. 3463-3472· 0 citations
Abstract
Despite the remarkable capabilities of Large Vision-Language Models (LVLMs), they remain vulnerable to jailbreak attacks. The integration of visual modalities further expands the attack surface due to its high-dimensional nature. To ensure the secure development of LVLMs, it is essential to develop more potent jailbreak attacks to probe their safety boundaries. This work focuses on universal attacks as they better reflect real-world scenarios. However, achieving robust universality remains challenging due to the tendency to overfit specific semantics within the training corpus. To bridge this gap and enhance generalization across diverse instructions, we propose Learning Universal Semantic Simplex (LUSS), a novel jailbreak method that shifts the optimization paradigm from point-wise perturbations to continuous adversarial simplices. By incorporating a sharpness-aware penalty and a simplex-based update strategy, LUSS captures a broader range of adversarial semantics within a convex hull to mitigate overfitting. To facilitate more rigorous evaluation, we introduce HarmInstBench, a manually curated benchmark with hierarchical annotations for diverse malicious scenarios. Extensive experiments on multiple LVLMs, including MiniGPT-4, LLaVA-v1.5, and Qwen2.5-VL, demonstrate that LUSS significantly outperforms state-of-the-art baselines, achieving up to a 60.8% relative improvement in Attack Success Rate (ASR), 31.8% in toxicity score, and a 44.7% gain in Malicious Intent Fulfillment Rate (MIFR). Warning: this paper contains harmful text!
It is concluded that while visual perturbations act as a viable near-term stopgap, long-term assessment security requires educators to reapproach assessment design given continually increasing VLM performance.
Christopher Burger, Christina Trotter, Joseph D. Carlisle et al.· arXiv.org· 0 citations
Ensuring trustworthiness and versatility in Open-World Visual Recognition is a core goal of Multimodal Large Language Models (MLLMs). While integrating segmentation is desirable, prevailing methods require invasive fine-tuning, which often leads to catastrophic forgetting of the model’s general-purpose capabilities. Th...
Jia-Zhen Liu, Long Chen· International Journal of Com...· 8 citations
Although Large Vision-Language Models (LVLMs) have shown great capabilities across various downstream tasks, they are proven to be vulnerable to adversarial examples, thus resulting in critical safety issues. Existing adversarial attacks against LVLMs predominantly rely on uniform, image-wise perturbations, while large...
Dai-Zong Liu, Tian-Yao Luo, Junhao Dong et al.· IEEE Transactions on Pattern...· 0 citations
Multimodal large language models (MLLMs) are vulnerable to visual jailbreak attacks that bypass safety safeguards. Existing methods typically optimize isolated adversarial examples, limiting attack diversity, transferability, and generalization. We propose Jailbreak Connectivity (JC), a framework based on the observati...
Xu Zhang, Zi-Qing Hu, Shuo Han et al.· 0 citations
ABench is a fine-grained, handcrafted benchmark specifically tailored to taxonomized values in Chinese, grounded in a rigorous hierarchical taxonomy comprising 158 derived value subcategories and utilizing question-specific scoring criteria for nuanced assessment.
Li-Ping Yang, Yong-Qi Leng, Jing-Ting Zheng et al.· Proceedings of the 2026 Conf...· 0 citations