Deepfake images: swapped, edited and generated stills compared

This page is part of the deepfake detection hub. It extends the video-centred face swap versus reenactment comparison to the single frame, where there is no motion, no audio and no second frame to compare against.

01What are the three kinds of deepfake image?

DF40, the 2024 benchmark of 40 manipulation techniques, sorts them into four types: face swapping (10 methods), face reenactment (13), entire face synthesis (12) and face editing (5) (arXiv 2406.13495). For a still image the middle two collapse into one, because reenactment of a single frame is an edit of expression or mouth shape, and the four become three: swapped, edited and generated.

A swapped image starts from a real photograph and replaces the face region with another identity, then blends the result back into the original background. An edited image keeps the identity and changes something about it: expression, age, hair, or the mouth shape in a reenacted frame. A generated image has no source photograph at all; a network such as StyleGAN2, trained on the 70,000 image FFHQ set, produced every pixel (arXiv 1912.04958). The generated faces page covers the third kind on its own.

02Which trace does each kind leave?

The published detectors are built around one trace each, and the trace follows from how the image was made.

Face X-ray's authors observe that "most existing face manipulation methods share a common step: blending the altered face into an existing background image", and their detector outputs a greyscale map "showing the blending boundary for a forged image and the absence of blending for a real image" (arXiv 1912.13458). For generated images, Frank and colleagues report that "in frequency space, GAN-generated images exhibit severe artifacts that can be easily identified", and that these are "caused by upsampling operations found in all current GAN architectures" (arXiv 2003.08685). Edits confine the change to a small region: FaceForensics++ describes NeuralTextures as modifying only the mouth region, and a detector has to find one small area that disagrees with the rest of a real face (arXiv 1901.08971).

03Does a detector built for one kind catch the others?

Not reliably, and DF40 measured it. Its Table 3 trains an Xception detector on face swaps in the FaceForensics++ domain and tests it on each type: 0.991 AUC on other face swaps, 0.892 on reenactments and 0.810 on entire-face synthesis. Its first finding is that the drop is asymmetric: training on swaps transfers to whole-face synthesis at about 0.8, while training on synthesis transfers back to swaps at about 0.6. The authors' reading is that "fake regions matter": a swap teaches a detector about a face-shaped patch inside a real frame, and a generated image has no such patch.

The reverse holds for generated images. Wang and colleagues found that a classifier trained on one GAN, ProGAN, "is able to generalize surprisingly well to unseen architectures, datasets, and training methods" among CNN generators (arXiv 1912.11035), but that generalisation is across generators of one kind, not across the three kinds on this page. The AI image detection page follows that line of work into diffusion models.

04What happened on circulated images?

Deepfake-Eval-2024 collected 1,975 images from social media and detection-platform users during 2024: 767 real and 1,208 fake, from 88 websites in 52 languages (arXiv 2503.02857). It ran three open-source image detectors off the shelf and reports, in its Table 3, each model's AUC on the new set beside its AUC on the data it was published with: UFD 0.56 against 0.94, DistilDIRE 0.52 against 0.99, NPR 0.53 against 0.98. The paper's summary across modalities is "an average drop in AUC of 50% for video, 48% for audio, and 45% for image models".

Two further rows matter. The best of eight anonymised commercial image models scored 0.82 accuracy and 0.90 AUC (Table 5), with precision 0.99 and recall 0.71, so it missed about three fakes in ten while almost never flagging a real one. Fine-tuning the open-source models on 60 percent of the set and testing on the rest raised NPR to 0.73 AUC but left UFD and DistilDIRE at 0.56 (Table 4); the average image gain was 4.5 percent, against 57.6 percent for video and 80.6 percent for audio. Images were the modality where seeing the new distribution helped least.

05Why do images lose more than they gain?

A still image offers a detector less than a video does. There is no temporal inconsistency, no lip and audio mismatch and no second frame to average over, so every decision rests on pixel statistics that a platform's re-encoding alters. FaceForensics++ measured that effect on frames: its XceptionNet fell from an accuracy of 99.26 percent on raw frames to 81.00 percent on the same frames at the heavier H.264 setting. The public deepfake datasets compared table shows where each set's stills come from; FaceForensics++'s 1.8 million images are frames cut from its 4,000 fake videos, so a detector trained on them has learned video compression rather than the JPEG pipeline of a shared photograph.

06How this fits the rest of the site

The deepfake AI on both sides page explains why the generators that make these three kinds and the classifiers that catch them share pretrained networks and datasets. The evidence order for an AI generated image page turns the figures here into a reading order: with image detectors near chance on circulated fakes, provenance and context carry more weight than a score. Swap generators measure identity with recognition networks such as the one described on the ArcFace angular margin loss page, which is why the identity in a good swap survives a verification check even when the blending boundary does not.