Skip to content
Book Open access

From Forensic Masks to Ranked Reasons: Evidence-Guided Multimodal Explanation for Document Image Tampering

Nov 2026 · Proceedings of the 2nd Deepfake Forensics Workshop: Detection, Attribution, Recognition, and Adversarial Challenges in the Era of AI-Generated Media · 0 citations · 8 references

Abstract

Document images, including scanned archives, contracts, certificates, and administrative records, are frequently used as digital evidence. Local edits to characters, seals, dates, or handwritten fields can change their meaning while leaving only subtle forensic traces. Existing localization models can predict suspicious regions, but their masks are difficult to interpret; general-purpose multimodal large language models (MLLMs) can describe document content, but often miss low-level tampering evidence or hallucinate unsupported explanations. We present an evidence-guided multimodal framework that turns forensic localization masks into structured, ranked document tampering explanations. A localization front-end predicts a tampering probability map, which is binarized and converted into coordinate prompts. A LoRA-tuned Qwen3-VL-4B model then generates a structured response containing tampering judgment, localization, evidence-based rationales, and rationale ranking. We construct 6,000 instruction-tuning samples from archival document tampering images and DocTamper. Under the present task-level setting, the proposed model achieves 0.794 accuracy, 0.802 F1, 0.725 cosine semantic similarity (CSS), and 0.518 NDCG@5, with higher scores than the evaluated zero-shot MLLM references, including LLaVA, InternVL2, Qwen2-VL, and GPT-4o. Additional localization, ablation, and cross-domain analyses support the use of document-specific frequency-aware evidence within the same replaceable mask-prior interface.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.