Skip to content
Open access

JPEG AI image compression visual artifacts: detection methods and dataset

Nov 2024 · Multimedia Systems · Vol 32 · 10 citations · 61 references
Computer Science

Abstract

Learning-based image compression has been outperforming traditional codecs in recent years. However, despite their overall efficiency gains, neural network-based codecs may still produce localized perceptual distortions on individual images. We propose a set of methods to detect the three most common types of neural image compression artifacts: texture or boundary degradation, color change, and text corruption. The proposed methods localize regions in neural compressed images affected by artifacts and quantify their strength. We consider only regions distorted by the neural codec but correctly reconstructed by a traditional codec at the same bitrate. Using these methods, we first collect artifacts for the JPEG AI reference software and its early unpublished versions relative to HM-18.0, the H.265 reference software, by processing approximately 350,000 unique images from the Open Images dataset across various compression-quality parameters. Since neural artifacts follow common patterns, we then apply the same protocol to five additional learned codecs (Cheng 2020, EVC, ELIC, STF, QRes-VAE), which allows us to verify that the proposed detection pipeline generalizes beyond JPEG AI. In total, the resulting dataset contains 50,370 artifacts validated through crowdsourced subjective evaluation. Our results show that neural codecs introduce localized distortions of a different nature than classical compression artifacts and that such distortions are often insufficiently captured by standard full-reference image quality metrics. In contrast, the proposed artifact-oriented methods detect these failure cases more reliably. The proposed dataset of images that exhibit neural compression artifacts along with the proposed detection methods can be used to analyze, debug, and improve future neural image compression methods. We release the source code for the proposed methods and a subset of our dataset at https://github.com/msu-video-group/jpeg-ai-artifacts.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.