FragFake: A Dataset for Fine-Grained Detection of Edited Images with Vision Language Models
Journal:
arXiv
Published Date:
May 21, 2025
Abstract
Fine-grained edited image detection of localized edits in images is crucial
for assessing content authenticity, especially given that modern diffusion
models and image editing methods can produce highly realistic manipulations.
However, this domain faces three challenges: (1) Binary classifiers yield only
a global real-or-fake label without providing localization; (2) Traditional
computer vision methods often rely on costly pixel-level annotations; and (3)
No large-scale, high-quality dataset exists for modern image-editing detection
techniques. To address these gaps, we develop an automated data-generation
pipeline to create FragFake, the first dedicated benchmark dataset for edited
image detection, which includes high-quality images from diverse editing models
and a wide variety of edited objects. Based on FragFake, we utilize Vision
Language Models (VLMs) for the first time in the task of edited image
classification and edited region localization. Experimental results show that
fine-tuned VLMs achieve higher average Object Precision across all datasets,
significantly outperforming pretrained models. We further conduct ablation and
transferability analyses to evaluate the detectors across various
configurations and editing scenarios. To the best of our knowledge, this work
is the first to reformulate localized image edit detection as a vision-language
understanding task, establishing a new paradigm for the field. We anticipate
that this work will establish a solid foundation to facilitate and inspire
subsequent research endeavors in the domain of multimodal content authenticity.
