Skip to content
Log in
Conference Open access

SecureBreak: A Dataset towards Safe and Secure Models

Marco Arazzi Vignesh Kumar Kembu Antonino Nocera
2026 · Proceedings of the 15th International Conference on Data Science, Technology and Applications · 0 citations · 29 references

TL;DR

SecureBreak is introduced, a safety-oriented dataset designed to support the development of AI-driven solutions for detecting harmful LLM outputs caused by residual weaknesses in security alignment and is valuable not only for constructing post-generation filtering modules that act as a last-line defense, but also for building additional supervisory intelligence for alignment optimization.

Abstract

: Large language models are becoming pervasive core components in many real-world applications. As a consequence, security alignment represents a critical requirement for their safe deployment. Although previous related works focused primarily on model architectures and alignment methodologies, these approaches alone cannot ensure the complete elimination of harmful generations. This concern is reinforced by the growing body of scientific literature showing that attacks, such as jailbreaking and prompt injection, can bypass existing security alignment mechanisms. As a consequence, additional security strategies are needed both to provide qualitative feedback on the robustness of the obtained security alignment at the training stage, and to create an “ultimate” defense layer to block unsafe outputs possibly produced by deployed models. To provide a contribution in this scenario, this paper introduces SecureBreak , a safety-oriented dataset designed to support the development of AI-driven solutions for detecting harmful LLM outputs caused by residual weaknesses in security alignment. The strong reliability of the proposed dataset derives from the adopted manual annotation procedure, in which labels are assigned conservatively to prioritize safety even in the presence of minor disagreements in the annotators’ opinion. Our exploratory data analysis campaign shows satisfactory performance in the detection of unsafe content across several risk categories. To evaluate its effectiveness, we measure the performance of several pre-trained LLMs in the considered classification setting under baseline conditions and compare these results with those obtained after fine-tuning the same models on SecureBreak . The results indicate that the dataset is valuable not only for constructing post-generation filtering modules that act as a last-line defense, but also for building additional supervisory intelligence for alignment optimization. We show that models fine-tuned on SecureBreak improve safety classification by up to 20% over baselines, with small models reaching up to 90.14% accuracy and sometimes outperforming larger models. In particular, classifiers derived from SecureBreak can be used to measure residual safety failures, inform whether additional training or refinement steps are necessary, and ultimately support more controlled and effective security alignment workflows.

DOI Semantic Scholar
Read PDF

Similar papers

Book Open access Aug 2026

The 2nd SeT-LLM Workshop on Secure and Trustworthy Large Language Models

Large language models (LLMs) are increasingly embedded as core components of data-centric systems, supporting analytical decision making, and automated reasoning over large-scale, heterogeneous datasets. Yet their deployment in open-world environments raises fundamental challenges to security and trustworthiness: LLMs can leak sensitive data, fall prey to prompt injection and jailbreaks, generate misinformation, and behave unpredictably under adversarial inputs, failures that propagate through data pipelines and affect downstream decisions. The rise of LLM-based agents further amplifies these risks through unsafe tool use and autonomous decision-making. The 2nd SeT-LLM Workshop on Secure and Trustworthy Large Language Models brings together researchers and practitioners from data mining, machine learning, security, and responsible AI to address these issues from a data-centric, system-level perspective, spanning robust defenses, trustworthy evaluation, privacy and copyright protection, robustness, alignment and safety, agent security, and high-stakes applications. Through invited talks, contributed papers, a poster session, and a panel discussion, the workshop prioritizes early-stage ideas, system experiences, and open problems across the lifecycle of LLM-based systems.

Lu Lin, Jinghui Chen, Ting Wang et al. · 0 citations
Preprint Aug 2026

Generating Attacks for LLMs with GFlowNets

This study proposes an automated, human-independent, and adaptive approach leveraging GFlowNets to identify LLM vulnerabilities by utilizing one large language model to test another, and introduces a model capable of generating attack inputs in the Turkish language.

Berkay Ozcam, Irem Onen, M. Amasyalı et al. · 0 citations
Conference Jul 2026

Generating Attacks for LLM with GFlowNets

The rapid advancement of Large Language Models (LLMs) has facilitated their ubiquitous integration into various domains, leading to widespread adoption. However, this escalating trend has introduced significant security vulnerabilities, necessitating the identification and mitigation of flaws arising from malicious exploitation. Red teaming assessments, conducted to evaluate model robustness through diverse adversarial inputs, are essential for exposing security risks and implementing countermeasures. Currently, red teaming is performed either manually by experts or automatically using predefined attack datasets. Nevertheless, manual testing remains time-consuming, while existing automated methods suffer from limited creativity due to their inherent dependency on fixed datasets. In this study, we propose an automated, human-independent, and adaptive approach leveraging GFlowNets to identify LLM vulnerabilities by utilizing one large language model to test another. Within this framework, an attacker model is trained against a specified victim model to perform automated red teaming and provide a quantitative robustness score. This research aims to generate more effective adversarial attacks in English compared to existing benchmarks and, as a novel contribution to the literature, introduces a model capable of generating attack inputs in the Turkish language.

Berkay Özçam, İrem Önen, E. I. Tatli et al. · 0 citations
Open access Aug 2026

DT-GenShield: A Digital Twin-Driven Runtime Security Architecture for Protecting Large Language Models Against Indirect Prompt Injection

Large Language Models (LLMs) are increasingly deployed in security-critical applications but remain vulnerable to indirect prompt injection attacks that cannot be fully addressed by conventional prompt detection techniques. This paper proposes DT-GenShield, a Digital Twin-driven runtime security architecture that integrates semantic threat detection, operational state representation, policy-guided mediation, and runtime logging to protect LLM-based systems before model inference. The proposed architecture was evaluated using the LLMail-Inject, BIPIA, and LLM-PIEval benchmarks under matched baseline and DT-mediated execution. Experimental results across the LLMail-Inject, BIPIA, and LLM-PIEval benchmarks demonstrate consistent improvements in runtime protection under the evaluated experimental settings. On the LLMail-Inject benchmark, DT-GenShield reduced the Attack Success Rate (ASR) from 9.50% to 5.50%, corresponding to a relative reduction of 42.11%, while additional validation on the BIPIA benchmark confirmed effective suppression of malicious prompt propagation with minimal impact on legitimate requests. These findings demonstrate that Digital Twin-driven runtime mediation extends prompt injection defense beyond standalone detection and provides an effective architecture for securing LLM-based systems against emerging semantic attacks.

Alaa Alnemari, Mashael M. Alsulami · 0 citations