Capable Enough to Be Hijacked: Quantization, Caution, and Indirect Prompt Injection in Small Edge LLMs

Abstract

Small language models are increasingly quantized to run offline on laptops and edge devices, where they often read untrusted documents. This preprint asks whether this compression changes their safety behaviour. We evaluated Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct at eight llama.cpp GGUF precision levels (F16 to Q2_K) on 1,175 prompts: harmful requests from JailbreakBench, HarmBench and AdvBench; benign requests; over-refusal probes from XSTest; and a new benchmark of 375 business documents (CVs, emails, supplier invoices, product reviews and IT support tickets), 300 containing a hidden instruction and 75 clean twins. All 18,800 responses were generated on a consumer laptop CPU and scored with deterministic rules validated against human labels (Cohen's κ = 0.86 for refusal, 0.79 for injection outcome). Three findings emerge. (1) The more capable 1.5B model was easier to hijack: 30.0% indirect prompt injection success versus 21.8%, and 72.2% versus 11.5% for task-hijacking attacks. (2) Quantization did not erode refusal of harmful requests but increased refusal of harmless ones; at Q2_K the 1.5B model refused 64.7% of benign requests versus 30.7% at F16. (3) Effects were format-specific: Q4_K_M was statistically indistinguishable from F16 on every tested metric, while the similarly sized Q4_0 was not. Models almost never flagged injected instructions. Code, benchmark and per-response labels: https://github.com/Chuka-J-Uzo/edge-llm-quantization-injection