AI deepfakes: how AI makes them and how AI is used to find them

This page belongs to the deepfake detection hub. It does not repeat the definition on the what is a deepfake page; it sets the making side and the finding side next to each other and shows where they share networks, training data and benchmarks.

01Which AI makes a deepfake?

Three families of generative model have produced the fakes that benchmarks test. The first is the autoencoder pair behind the original DeepFakes method: FaceForensics++ describes it as "two autoencoders with a shared encoder" trained on a source and a target face, with the source decoder applied to the target (arXiv 1901.08971). The second is the generative adversarial network, whose face line runs from StyleGAN to StyleGAN2, trained on the 70,000 image FFHQ set and reaching an FID of 2.84 on it (arXiv 1912.04958). The third is the diffusion model, which DF40's authors list among the 12 entire-face-synthesis methods in their 40 technique set, with PixArt-alpha from 2024 as the latest (arXiv 2406.13495).

DF40 groups all 40 techniques into four types: face swapping (10 methods), face reenactment (13), entire face synthesis (12) and face editing (5). The first two edit a real video of a real person; the third produces a face that belongs to nobody; the fourth changes attributes such as age or expression in a real photograph. The kinds of deepfake stills page compares the traces each type leaves in a single frame.

02What does the generator borrow from face recognition?

A face swap has to keep the source person's identity while it adopts the target's pose, lighting and expression. Modern swap generators measure identity with the same networks that recognition systems use. SimSwap's identity injection module takes its identity vector from a pretrained ArcFace model, and its authors describe the goal as "transferring the identity of an arbitrary source face into an arbitrary target face while preserving the attributes of the target face" (arXiv 2106.06340). FaceShifter's first stage uses "a pretrained state-of-the-art face recognition model" as its identity encoder, and its evaluation scores identity retrieval with a second recognition network (arXiv 1912.13457).

That is the first shared component. The loss that trains ArcFace, and why a results table names it as a backbone, is explained on the ArcFace angular margin loss page. The point for this page is that the recognition network sits inside the generator as a judge of whether the swap worked, which means the generator is trained to satisfy exactly the kind of network a verification system would run.

03Which AI finds a deepfake?

The detectors in the main published benchmark are image classifiers. DeepfakeBench's naive detectors are Xception, EfficientNet-B4, MesoNet and a ResNet with augmentation, and eight of its fifteen detectors keep an Xception backbone and change only the input or the loss (arXiv 2307.01426). The benchmark leverages "pre-trained backbones from ImageNet if feasible", and its ablation finds that pre-training is worth about 10 percent AUC for Xception on the DFDC preview set. Its authors trace part of Xception's edge over ResNet34 to the depthwise separable convolution module that ResNet lacks.

DF40 adds a newer kind of backbone: CLIP, the image and text model, which its Finding-3 states "excels in deepfake detection than other baselines (such as Xception)". The face recognition backbones in detectors page lists which published detectors reuse which network. None of the fetched papers reuses a generator's own architecture as a detector; what both sides share is the family of pretrained vision networks and the aligned 112 by 112 or 256 by 256 face crop they operate on.

04Why are both sides scored on the same datasets?

A generator paper proves its fakes are good by showing that a detector or a recognition network is fooled; a detector paper proves its method is good by catching those same fakes. Both therefore need a common pool of real footage with known manipulations, and the field has settled on a few. The table lists the sets that appear on both sides of the literature fetched for this page.

The scores on the published results table all come from the right-hand column, and the conditions in each row (training set, test set, compression) are the conditions the left-hand column fixed years earlier.

05How much does the detector lose when the generator changes?

The benchmarks measure this directly. In DeepfakeBench Table 3 the Xception detector trained on FaceForensics++ c23 scores 0.9637 on that set, 0.8261 on the same fakes compressed to c40, and 0.7718 averaged over eight datasets it never saw. In DF40 Table 3 an Xception trained on face swaps in the FaceForensics++ domain scores 0.991 on other face swaps, 0.892 on reenactments and 0.810 on entire-face synthesis; in Table 5, on fakes from unknown methods and unknown footage, its average is 0.546, with CLIP at 0.692. DF40's authors summarise the state of the art as Finding-2: "Existing well-designed (SoTA) detectors may not have obvious advantages over the baseline".

The in-the-wild figure is lower still. Deepfake-Eval-2024, built from fakes that circulated in 2024, reports "an average drop in AUC of 50% for video, 48% for audio, and 45% for image models" for open-source detectors against their original benchmarks (arXiv 2503.02857). No single benchmark score predicts how a detector performs on manipulations it has not seen.

06Does an AI detector beat a person?

On the fakes a detector was trained for, yes, by a wide margin. FaceForensics++ ran a user study with 204 participants who reached 68.69 percent on raw video and 58.73 percent on low quality video, while XceptionNet trained on all four methods reached an accuracy of 99.26 percent raw and 81.00 percent at low quality. On fakes nobody trained for, the gap closes. Groh and colleagues (PNAS 2022) found that the leading DFDC model scored 65 percent on the 4,000-video holdout set and that recruited participants scored 66 percent on the holdout videos drawn for the study, "similarly accurate, while making different kinds of mistakes". On synthetic still faces, Nightingale and Farid (PNAS 2022) measured 48.2 percent human accuracy against 50 percent chance.

07What the shared toolkit means for reading a score

A detector score is a statement about one network, one training set and one test set. Because the generator side and the detector side draw on the same pretrained vision models and the same datasets, a new generator trained against a current recognition network is also trained, in effect, against the features a current detector relies on. The evidence order for an AI generated image page puts a detector score after provenance and context for that reason. The face recognition section covers the recognition side of the same pipeline.