From Forensic Masks to Ranked Reasons: Evidence-Guided Multimodal Explanation for Document Image Tampering
Abstract
Document images, including scanned archives, contracts, certificates, and administrative records, are frequently used as digital evidence. Local edits to characters, seals, dates, or handwritten fields can change their meaning while leaving only subtle forensic traces. Existing localization models can predict suspicious regions, but their masks are difficult to interpret; general-purpose multimodal large language models (MLLMs) can describe document content, but often miss low-level tampering evidence or hallucinate unsupported explanations. We present an evidence-guided multimodal framework that turns forensic localization masks into structured, ranked document tampering explanations. A localization front-end predicts a tampering probability map, which is binarized and converted into coordinate prompts. A LoRA-tuned Qwen3-VL-4B model then generates a structured response containing tampering judgment, localization, evidence-based rationales, and rationale ranking. We construct 6,000 instruction-tuning samples from archival document tampering images and DocTamper. Under the present task-level setting, the proposed model achieves 0.794 accuracy, 0.802 F1, 0.725 cosine semantic similarity (CSS), and 0.518 NDCG@5, with higher scores than the evaluated zero-shot MLLM references, including LLaVA, InternVL2, Qwen2-VL, and GPT-4o. Additional localization, ablation, and cross-domain analyses support the use of document-specific frequency-aware evidence within the same replaceable mask-prior interface.