Impersonation Ambiguity as a Security Signal: A Quantitative Framework for Role-Play vs. Malicious Impersonation Detection in Large Language Models

Abstract

Large language models (LLMs) increasingly support persona adoption and character role-play as a first-class, legitimate usecase; creative writing assistants, companion chatbots, and educational simulations all depend on it. Yet the same mechanism isthe substrate of one of the most persistent jailbreak families in the literature, and it is exploited directly for social-engineering-style impersonation of real individuals, brands, and authorities. Prior security systems typically treat “impersonation” as asingle binary label, and our own earlier work on the SemGuard gateway found empirically that this label is the most contestedcategory any moderation system can assign: three independent frontier-model judges agreed on only 3 of 166 generated imper-sonation examples (a 98.2% inter-judge rejection/disagreement rate), far higher than any other threat category in that study.This paper treats that disagreement not as annotation noise to be minimized away, but as a measurable security signal in itsown right. We formalize the distinction between benign role-play (persona adoption bounded by fictional framing, consent, andnon-deceptive intent) and malicious impersonation (persona adoption intended to deceive a third party, extract value, or evadeattribution) as a multi-dimensional classification problem rather than a single decision boundary. We propose a quantitativeframework – the Impersonation Ambiguity Index (IAI) – that scores any persona-bearing input along four independent axes:target realism, deceptive intent, consent/context boundedness, and downstream actionability. We show how this index can beoperationalized on top of an LLM-as-Judge annotation pipeline, propose an evaluation protocol using multi-judge Fleiss’ κ peraxis rather than per label, and outline how the resulting ambiguity scores can be fed back into a semantic security gateway (inthe style of Triple-Anchor architectures) as a fifth, uncertainty-aware anchor rather than forcing a premature binary decision.We situate this proposal against the jailbreak, persona-prompting, and AI-enabled social-engineering literatures, and arguethat ambiguity-aware, axis-decomposed labeling is a necessary next step for any multilingual or cross-cultural moderation sys-tem, where role-play norms and what counts as “impersonation” vary sharply across languages and communities. This paperpresents the framework and evaluation protocol together with an initial n = 40 pilot run (Section V-E); full-scale validation onthe expanded, dialect-stratified dataset remains future work.