STeREx-Net: Diffusion-Residual Evidence Fusion for Explainable Three-Class Synthetic-Media Forensics
TL;DR
STeREx-Net should be viewed as a human-supervised forensic research framework with explicitly characterized robustness, localization, and generalization boundaries rather than as a universally robust detector.
Abstract
AI-generated and locally manipulated images can support impersonation, forged evidence, identity-document abuse, and other forms of digital fraud, making reliable content-authenticity analysis increasingly important. This paper presents STeREx-Net, a three-class forensic framework for distinguishing real, fully synthetic, and locally tampered images using frozen diffusion-derived residual evidence, spatially aligned RGB features, multi-task prediction heads, and reviewable visual evidence. On the official balanced 60,000-image SID-Set test, the original model achieved 96.24% accuracy, 96.26% macro F1, and a multiclass Matthews correlation coefficient of 0.944. Separate matched three-seed ablations showed that diffusion-residual evidence is materially useful relative to RGB-only input; however, residual-only and simple-fusion controls outperformed the proposed fusion on the clean SID-Set, so fusion superiority is not claimed. Frozen robustness testing further revealed strong condition dependence: SID-Set macro F1 decreased from 0.9655 on clean images to 0.7987 under JPEG Q75 and 0.6330 under JPEG Q50, while generator-stratified AIS-4SD results also varied substantially. Zero-shot transfer to FantasyID failed to detect tampered samples, whereas leakage-safe restricted adaptation partially recovered tampered recall to 0.3447 and reduced the expected calibration error from 0.7815 to 0.2077, at the cost of lower real-image recall. A validation-selected localization intervention increased Dice from 0.2817 to 0.3413 but remained precision-biased and non-uniform across manipulation sizes. Quantitative explanation analysis supported in-domain decision faithfulness and benign-transformation stability, but these properties did not transfer consistently to external data. STeREx-Net should therefore be viewed as a human-supervised forensic research framework with explicitly characterized robustness, localization, and generalization boundaries rather than as a universally robust detector.
