This work assesses indirect prompt injection in DeepSeek Harness, using AI-Infra-Guard (A.I.G) to construct tests, deliver controlled taint, execute DSH, collect traces, and judge outcomes, and relates these results to DSH's treatment of tool results, additional contexts, and tool-call policy hooks.
Abstract
We assess indirect prompt injection in DeepSeek Harness (DSH), using AI-Infra-Guard (A.I.G) to construct tests, deliver controlled taint, execute DSH, collect traces, and judge outcomes. The study covers 14,560 controlled executions over 16 indirect-content channels, text and file carrier modes, 35 payload objectives, one unmodified baseline, and 12 attack methods. The experiment preserves DSH's agent loop, tool registry, model adapter, and session-event path; source tools and sensitive sinks are local fixtures, so attempted actions are recorded without external side effects. We evaluate each trace with a deterministic rule-based judge, \JudgeR{} (RuleJudge), and a semantic LLM-based judge, \JudgeL{} (LLMJudge). The strongest observed attack success rates are 17.0% under \JudgeL{} for fake-completion attack in text mode, 25.5% under \JudgeR{} for hidden Unicode in file mode, and 16.0% under \JudgeR{} for the skills channel in file mode. \JudgeL{} also assigns partial compliance more often than \JudgeR{} (7.3% versus 2.0%). We relate these results to DSH's treatment of tool results, additional contexts, and tool-call policy hooks, then identify controls that should sit between untrusted content and sensitive actions. Our code is available at https://github.com/Tencent/AI-Infra-Guard/tree/main/Research/deepseek-harness-security-assessment .
It is argued that evaluation validity is a prerequisite for, not a footnote to, defense claims in agentic security, and a harness is provided to provide an instrument that enforces it.
Indirect prompt injection embeds malicious instructions within external content retrieved by LLM-based agents, altering target behavior without user authorization. We introduce pikit, a research toolkit designed to systematically evaluate these threats across three core dimensions: attacks (13 methods), channels (16 ca...
Zong-Hao Ying, Xiang-Fan Wu, Bo Yang et al.· 0 citations
This work presents HarnessSecurity, the first systematic empirical study and benchmark of open- and closed-source coding agent harnesses, and derives a ten-mechanism taxonomy and assesses 400 harness-mechanism cells using independent ratings by researchers and large language model judges.
Zheng-Yang Zhu, Li-Ming Huang, Run-Min Ji et al.· 0 citations
The findings show that defense evaluation should report attack outcomes, goal feasibility, legitimate-task utility, action availability, operational measures, and run-to-run variation.
Adil Khan, Khaled AlKhanbashi, Azza Mohamed· Computers· 2 citations
A systematic review and structured descriptive synthesis of research on defenses against prompt-based attacks in language model and agent systems reveals trade-offs between security effectiveness, performance, and system complexity as well as major gaps in benchmarks, indirect attack coverage, and multi-agent evaluatio...
Sana Mourad, E. E. Abdallah, Mohammad Ababneh· Electronics· 0 citations
A tool-using LLM agent that reads attacker-controlled web content while holding a secret faces indirect prompt injection: the content may make it exfiltrate the secret, which may make it exfiltrate the secret.
Md Habibur Rahman, Jaehoo Kim· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.