Types of deepfakes: the four kinds, their datasets and their cues in one table

This page is the compact taxonomy for the deepfake detection hub: one table, then short notes. The four kinds of face deepfake page explains each kind at length, and the generation methods catalogue lists all 40 DF40 techniques.

01The four types in one table

Counts per dataset come from DF40's comparison table (arXiv 2406.13495); the Celeb-DF figures from its repository and paper (arXiv 1909.12962); the human study rows from FaceForensics++ (arXiv 1901.08971) and Nightingale and Farid (PNAS 2022). The public deepfake datasets table holds the full counts for every set named.

02How are the four types defined?

DF40 defines each type by what it changes in a target face. Face swap "aims to replace the content" of the target "with that of" the source while preserving the source identity. Face reenactment "preserves its identity" but has "its intrinsic attributes, e.g., pose, mouth and expression manipulated". Entire face synthesis "generates a entirely synthesis face". Face editing has "its external attributes altered, such as facial hair and age" (arXiv 2406.13495). The split between the second and fourth type is the split between moving a face and relabelling it.

FaceForensics++ used an older two-way split that still appears in papers: graphics-based methods (Face2Face, FaceSwap) against learning-based ones (DeepFakes, NeuralTextures). That split describes how a fake is made rather than what it changes, and the two cut across each other: FaceSwap and DeepFakes are both face swaps, one graphics-based and one learned.

03Which datasets cover which type?

Most public datasets cover one or two types. DF40's authors write that "most earlier public deepfake datasets published before 2021 generate deepfakes using only the face-swapping technique that involves a blending process", and that those datasets "contain only single or no more than 4 specific manipulation approaches". Their Table 1 counts FaceForensics++ at two swap and two reenactment methods, DFDC at five swap, one reenactment and two synthesis methods, and ForgeryNet at six, four, two and three across the four types.

ForgeryNet (CVPR 2021) is the only earlier set with all four types: 15 manipulation approaches, 121,617 fake video clips and 1,457,861 fake still images (arXiv 2103.05630). Celeb-DF v2 (CVPR 2020) is a single-type set of 5,639 face swap videos made from 590 YouTube originals of 59 subjects. DF40 (NeurIPS 2024) is the first to cover each type in depth, with 10, 13, 12 and 5 methods and over 2,000 detector evaluations across them.

04Does one detector cover all four types?

Not evenly. DF40's first finding is an "asymmetric performance drop among different forgery types": in its Table 3, Xception trained on face swaps scored a frame-level AUC of 0.991 on swaps, 0.892 on reenactment and 0.810 on synthesis, while Xception trained on synthesis scored 0.999 on synthesis but 0.665 on swaps. Training on the type with the smallest changed region transfers best, because a detector that learned to find a blended patch has learned something that a wholly generated face still violates in places, while the reverse is not true.

Face editing is the hardest transfer. In DF40's Protocol 3, the same swap-trained Xception averaged 0.546 across unseen editing and synthesis sets, with 0.426 on StyleCLIP; "white means 0.5 AUC (by chance)" in the paper's heat map. The published results page holds the per-set rows.

05Why does the type matter for reading a score?

A detection score is a score on one type unless the paper says otherwise. Most benchmark numbers come from FaceForensics++, which is swap and reenactment only, so a detector's headline AUC says nothing measured about synthesised or edited faces. The type also sets which cue survives the platform path: the detection on social media page shows that compression removes high-frequency traces first, which are the main cue for synthesised faces and a secondary one for swaps.

No single benchmark score predicts how a detector performs on a type it was not trained on. The same caution applies in face recognition, where a toolkit such as InsightFace lists training sets of different sizes and origins, and where a deployed system such as face recognition at airports is measured on its own gallery rather than on a public set.