Response-Only Adversarial Detection of LLM Outputs: A Comparative Study of Lexical, Contrastive, and Contextual Representations

Abstract

Large Language Models (LLMs) are increasingly used in settings where the original user prompt may be unavailable, redacted, or untrusted. This makes response-only adversarial detection an important but challenging problem, because the detector observes only the generated response while the label is determined by the originating prompt. In this study, we investigate response-only adversarial classification of LLM outputs through a comparative evaluation of lexical, contrastive, and contextual representations. Specifically, we compare TF-IDF features, triplet-loss-based metric embeddings, TF-IDF+triplet concatenation, and a Sentence-BERT contextual embedding baseline across three tasks: Prompt Type, Adversarial Type, and Adversarial Class. Experiments on 158 592 response records show that response-only detection is feasible but substantially more difficult than prompt-aware detection. Under stratified group 10-fold evaluation, the strongest TF-IDF+XGBoost baseline achieves 72.37% accuracy for Prompt Type, 82.06% for Adversarial Type, and 56.40% for Adversarial Class. Leave-one-LLM-out evaluation further shows that cross-model generalization remains challenging. Triplet-based metric learning provides model-dependent benefits, particularly for some Adversarial Type configurations, but it does not uniformly outperform strong lexical baselines. Low-dimensional embedding analysis shows that Insertion samples are relatively separable, whereas Prefix and Suffix samples substantially overlap. Overall, the findings provide a realistic benchmark for response-only adversarial detection and clarify the limits of contrastive representations when prompt information is absent.