Skip to content
Book Open access

Human Indirect Prompting Vulnerabilities and Intention-Aware Conversational Defenses for Large Language Models

Nov 2026 · Proceedings of the 35th ACM International Conference on Information and Knowledge Management · 0 citations · 15 references

Abstract

Large Language Models (LLMs) can explain sensitive topics across domains, but harmful goals may be concealed behind educational, fictional, historical, or research-oriented framing. We study Human Indirect Prompting (HIP), a conversation-level behavior in which a user steers an LLM toward restricted information without explicitly stating the unsafe objective. We construct a controlled, redacted dataset of 25 HIP prompts spanning five risk domains and ten prompting strategies, and evaluate five LLM families. Across 123 valid responses, indirect prompting achieves an overall attack success rate of 53.66% with substantial cross-model and cross-strategy variation. We then introduce Controller-Guided Intention-Aware Conversational Defense (IACD), a wrapper that uses structured experiment context and conversation-level control before the target model completes a request. In an ablation study using the most and least vulnerable target models from the five-model evaluation, direct harmful leakage is 88% and 20%, respectively, and falls to 0% under controller-guided conditions. Exact paired tests confirm significant reductions relative to direct prompting. These results characterize HIP and provide initial evidence for dialogue-level safety control, while the controlled use of benchmark intent metadata limits claims about autonomous intent inference.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.