Skip to content
Open access

Beyond Clean Benchmarks: A Reproducible Severity-Sweep Evaluation of AI-Generated Image Detector Robustness

Sep 2026 · Iconic research and engineering journals · 0 citations · 9 references

TL;DR

A concise, replicable evaluation of the two public detectors that function using differing architectures, which indicated that UnivFD was more accurate than CNNSpot in detection of original images and that no transformation crossed the threshold for practical significance set at ten.

Abstract

- AI-generated image detection technology is being adopted more frequently in journalism, content moderation, and digital-evidence verification, although the majority of AI detectors are tested on pristine, unaltered visuals and not on resized, recompressed, resampled, or altered formats one frequently encounters in the real world. Prior research, including the expansive multi-scenario RRDataset/RRBench (including seventeen detectors and more than ten vision-language models), has demonstrated that the performance of detectors deteriorates with image degradation; thus, this paper cannot claim to be the first work to show the robustness of detectors. Rather, it presents a concise, replicable evaluation of the two public detectors that function using differing architectures. — CNNSpot (a CNN created using ProGAN) and UnivFD (a frozen-CLIP semantic detector) were evaluated in a set of 2,000 images harmonized to derive a GenImage benchmark (Stable Diffusion v1.5, ADM, Midjourney, BigGAN). Both models were analyzed as frozen under a variety of conditions including JPEG compression of varying severity, image resizing and other manipulations. The results indicated that UnivFD was more accurate than CNNSpot in detection of original images (AUROC 0.741 vs. 0.662); this trend remains the same regardless of the 21 transformations. Gaussian blur was the only transformation that caused consistent monotonic drop in the performance. The JPEG compression, the resizing, and the noise led to minor changes that were non-monotonic. In addition, the cropping showed an improvement in AUROC scores for both detectors. Notably, however, no transformation crossed the threshold for practical significance set at ten. The compound chain of transformations did not hurt performance; where significant, compounding was mostly linked to the equal or better AUROC than the working of the worst.

Read PDF

Similar papers

Open access Oct 2026

Beyond Clean Benchmarks: A Reproducible Severity -Sweep Evaluation of AI Generated Image Detector Robustness

AI-generated image detectors are used often for verifying news, moderating content and handling evidence. However most tests for these detectors use images. Images that circulate in the world are often recompressed, resized, cropped or filtered. Large studies already show that detector accuracy drops when images are de...

Sunidhi, Ragini Rai, Pranay Tehra et al. · 0 citations

REALIS: A Curated Dataset for Studying the Challenges of AI Image Detection

REALIS provides a unified framework for measuring and improving the reliability of AI-image detectors under conditions that better reflect real-world use, and proposes a robustness protocol covering 35 transformations at five severity levels to analyze detector behavior under image processing.

A. Gushchin, Khaled Abud, G. Bychkov et al. · 0 citations
Open access Oct 2026

AI Image Detection Using an Ensemble of Efficientnet, Resnet and Vision Transformer

Modern generative models, such as GANs and diffusion models, produce highly realistic synthetic images that are virtually indistinguishable from authentic photographs, posing significant risks in forensics, journalism, and data verification. To address the need for a fast and accessible detection tool, this paper propo...

K. Dukhnovska, T. Holub · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.