Deepfake definition in research: how five papers scope the same word

This page belongs to the deepfake detection hub and sits beside the plain-language definition of a deepfake. That page answers the general question; this one is for readers of papers, who need to know what a given benchmark counted before they compare its scores with another.

01Why does the definition matter for reading a score?

A detector score is a statement about the fakes in one test set. If the test set holds only face swaps, the score says nothing about reenactment or about a synthetic face that belongs to nobody. The DF40 paper puts the point directly: "Most earlier public deepfake datasets published before 2021 generate deepfakes using only the face-swapping technique that involves a blending process", and such sets "contain only single or no more than 4 specific manipulation approaches" (arXiv 2406.13495).

So two papers can both report an AUC for "deepfake detection" and measure different tasks. The public dataset counts and licences table lists what each set contains; this page lists what each paper meant by the word. The two together tell a reader whether a published number transfers to their problem.

02How does Celeb-DF scope the word?

Celeb-DF scopes deepfake to one kind. Its abstract opens: "AI-synthesized face-swapping videos, commonly known as DeepFakes, is an emerging problem threatening the trustworthiness of online information", and the dataset "contains 5,639 high-quality DeepFake videos of celebrities generated using improved synthesis process" (arXiv 1909.12962). Every fake in the set is an identity swap of a public figure, made with the authors' own pipeline.

That narrow scope was the norm for its generation. The Celeb-DF paper itself sorts UADFV, DF-TIMIT and the FaceForensics++ DeepFakes subset as a first generation of deepfake datasets and Google's DFD, the DFDC preview and Celeb-DF as a second; all of them are face swap sets. A detector that reaches a high AUC on Celeb-DF has learned to find swapped identities, not moved mouths.

03How does FaceForensics++ scope it?

FaceForensics++ uses the word in two senses and says so. In its method section, "The term Deepfakes has widely become a synonym for face replacement based on deep learning", but "it is also the name of a specific manipulation method that was spread via online forums", which the paper writes as DeepFakes (arXiv 1901.08971). Its introduction then sorts all face manipulation into two families: "Current facial manipulation methods can be separated into two categories: facial expression manipulation and facial identity manipulation".

The benchmark follows that two-family scope. Its abstract states that it "is based on DeepFakes, Face2Face, FaceSwap and NeuralTextures as prominent representatives for facial manipulations at random compression level and size", with "a database of over 1.8 million manipulated images". DeepFakes and FaceSwap replace identity; Face2Face and NeuralTextures keep the identity and change the expression. The FaceForensics++ page covers the four methods and the compression levels; the reenactment methods explained page covers the second family on its own.

04How do DF40 and ForgeryNet scope it?

DF40 takes the widest scope of the five and defines it in four parts. Its abstract says "Deepfake techniques are commonly referred to as both face forgery and entire image synthesis. Most existing datasets only contain partial types of them" (arXiv 2406.13495). Section 2.1 then defines each type by what it changes: face swapping "aims to replace the content of x_t with that of x_s preserving the identity i_s"; face reenactment "preserves its identity i_t but has its intrinsic attributes a_t, e.g., pose, mouth and expression manipulated"; entire face synthesis "generates a entirely synthesis face"; face editing "has its external attributes b_t altered, such as facial hair and age". The dataset holds 10, 13, 12 and 5 methods of each, 40 in all. The paper's footnote adds the etymology: "Deepfake is a term derived from 'deep' (the use of deep nets) and 'fake' (does not exist)".

ForgeryNet (CVPR 2021) scopes by identity rather than by four kinds. Its abstract describes a "three-way (real / fake with identity-replaced forgery approaches / fake with identity-remained forgery approaches), and n-way (real and 15 respective forgery approaches) classification" over "2.9 million images, 221,247 videos" made with "7 image-level approaches, 8 video-level approaches" (arXiv 2103.05630). DF40's Table 1 sorts those 15 approaches as 6 face swap, 4 reenactment, 2 entire synthesis and 3 face editing, so ForgeryNet covers all four kinds under a two-way split of its own.

The kinds and method counts in the table come from DF40 Table 1 (arXiv 2406.13495); DFDC's row follows that table's reading of its 8 methods as 5 face swap, 1 reenactment and 2 entire synthesis.

05How does Deepfake-Eval-2024 scope it?

Deepfake-Eval-2024 drops the taxonomy and scopes by origin and medium. Its abstract describes "in-the-wild deepfakes collected from social media and deepfake detection platform users in 2024", amounting to "45 hours of videos, 56.5 hours of audio, and 1,975 images" from 88 websites in 52 languages (arXiv 2503.02857). The generator behind each item is unknown, so the authors cannot sort fakes by kind; they sort them by whether a forensic analyst judged them manipulated.

That scope is the broadest of the five, and it changes what a score means again. The same paper reports that open-source detectors' AUC fell by 50 percent for video, 48 percent for audio and 45 percent for images against their original benchmarks. A detector scored on Celeb-DF and the same detector scored on Deepfake-Eval-2024 were asked different questions, which is why the types table and the dataset table on this site always name the set beside the number.

06How do statutes define the word differently?

Legal definitions scope the word by effect on a viewer, not by method. Texas Penal Code 21.165 defines "deep fake media" as a depiction "created or altered through the use of software, machine learning, artificial intelligence, or any other computer-generated or technological means" that "appears to a reasonable person to depict a real person" doing something "that did not occur in reality". Minnesota Statutes 609.771 covers "any video recording, motion-picture film, sound recording, electronic image, or photograph" that is "so realistic that a reasonable person would believe" it. The state deepfake statutes table quotes each text; the deepfake meaning and origin page traces the dictionary entries. Identity verification uses a third vocabulary again: the ISO/IEC 30107 series speaks of presentation attacks and liveness, and the passive and active liveness checks page explains why an injected deepfake falls outside both.

For a researcher the difference is practical. A face edit that changes age, in DF40's scope, is a deepfake; under the Texas definition it is one only if it depicts an action that did not occur. A benchmark score and a statute therefore count different things even when they use the same word. The one rule that holds across all five papers and both statutes is that no definition is implied by the word alone: the paper or the section has to be read.