PRISM: A Lightweight Method for Jailbreak Detection via Prompt-Response Semantic Coherence Scoring in LLMs

Abstract

Detecting whether a jailbreak attack against a large language model (LLM) has succeeded is a challenging problem in AI security. Existing approaches rely on brittle string matching, expensive LLM-based judges, or task-specific classifiers that often fail to generalize. This paper investigates whether semantic coherence is sufficient to evaluate jailbreak success. In this work, we propose PRISM: Prompt-Response coherence for Identifying jailbreaks via Scoring Method, a lightweight method that models jailbreak detection as a coherence problem between the harmful prompt and the model response. The approach combines multiple semantic similarity signals within an ensemble of classical and simple machine learning models. Training in 3,480 pairs of prompt-responses from AdvBench, HarmBench and Alpaca, PRISM achieves F1 = 0.91 and AUC-ROC = 0.98 in the internal test set. In a held-out validation set of 694 labeled samples never seen during training, it outperforms six baseline attack success estimators, achieving the highest accuracy (0.746) and AUC-ROC (0.840), while being up to $\mathbf{1 6} \times$ faster than LLM-based methods. These results demonstrate that semantic coherence provides a robust and efficient signal for jailbreak detection, significantly reducing the implementation complexity of state-of-the-art approaches.