WildDeepfake: the first deepfake dataset built from fakes found online

The page belongs to the datasets cluster of the deepfake detection hub and is the dataset entry behind the broader deepfake detection in the wild page, which puts WildDeepfake beside the 2024 in-the-wild benchmarks. It is listed with its counts in the table of every public face deepfake dataset.

01What is WildDeepfake?

WildDeepfake was built by Zi, Chang, Chen, Ma and Jiang and published at ACM Multimedia 2020 (arXiv 2101.01456). Its premise is that every earlier detection dataset was made the same way: collect real videos, then run deepfake software on them. The authors call the result "virtual deepfakes" and argue they fail to represent what circulates online on two counts, "Lack of diversity" (few scenes, one person per scene, a handful of known methods) and "Low quality" (fakes made quickly, with visible flaws). WildDeepfake answers both by collecting the fakes themselves: "Both the real and deepfake videos in WildDeepfake were collected from the internet".

The released set holds 3,805 real and 3,509 fake face sequences, split into 6,508 for training and 806 for testing. The split was made on the similarity between sequences so that the test faces differ from the training faces. Because the clips were found rather than made, the dataset does not record which method produced each fake or how many people appear in total; the actor count column that other datasets fill is blank in the paper's own comparison table.

02How was the dataset collected and labelled?

The authors searched video sharing sites for titles containing "deepfake" and gathered over 1,200 candidate videos. They dropped videos whose titles and descriptions showed traditional face manipulation rather than a deepfake method, then searched for the real source video of each fake and dropped any fake without one. "This leaves us 707 well-made deepfake videos from the internet". Faces were found with the MTCNN detector, features extracted with an ImageNet pretrained MobileNetV2 to group them into sequences, and aligned with dlib facial landmarks.

Labelling took three trained annotators. Each labelled a face sequence real, fake or unknown from the video's title, located the source video for each fake, and marked a source as real only if it showed no obvious defects. Unknown sequences and any sequence the three annotators disagreed on were discarded. The paper notes that "labeling the 707 videos can take months". What remained was 1,180,099 face images in 7,314 sequences. Labels therefore rest on the uploader's title and the annotators' checks, not on knowledge of the generation pipeline; the how deepfakes are made page explains why that pipeline is otherwise what a dataset documents.

03How does it compare with lab datasets?

The paper's own comparison table puts the set beside five researcher-made datasets. WildDeepfake is not the largest by sequences, but it is the only one whose real and fake sources are both "Internet".

The counts are the paper's own; the Celeb-DF, DFDC and FaceForensics++ entries on this site's dataset table come from those datasets' papers and repositories and are quoted there with their own conditions.

04What did detectors score on it?

The authors ran eleven image-level networks and four sequence-level networks on each of six datasets, reporting accuracy. On the five lab sets the baselines did well: the best reached 90.02 percent on DeepfakeDetection, 99.65 and 99.91 percent on the two Deepfake-TIMIT splits, and 90.91 and 99.62 percent on the FaceForensics++ Deepfake low and high quality splits. On WildDeepfake "no baseline networks can achieve accuracy above 70%". XceptionNet, the strongest baseline on three of the five lab sets, fell from 99.62 percent on FaceForensics++ HQ to 69.25 percent.

Sequence-level networks did worse. In the paper's Table 4, P3D scored 53.20 percent, C3D 55.87, I3D 62.69 and the authors' ADDNet-3D 65.50 on WildDeepfake, each on clips of 50 frames at 112 by 112 pixels. The authors suggest that frame by frame generation also distorts the temporal signal, so that 3D networks have no clean motion cue to use. All figures are accuracy at the network's own threshold on a near balanced test set, which is a different quantity from the frame-level AUC most benchmarks report; the AUC score explained page sets out why the two are not interchangeable.

05What is ADDNet?

ADDNet, the Attention-based Deepfake Detection Network, is the authors' detector. Its idea comes from how swaps are made: most tools fuse the generated face into the frame through an attention mask over the face, so the detector builds a mask of its own. From 68 facial landmarks it draws a face mask and a second mask over the eyes, nose and mouth, blurs both, sums them and scales the result to the range 0 to 1. That mask is downsampled and multiplied into the intermediate feature maps of an "ADD block" that "follows an XceptionNet architecture", so the network weights the regions a generator most often touches. ADDNet-2D classifies single face images; ADDNet-3D runs several ADD blocks with shared weights over a sequence and feeds them to a 3D CNN.

The gain is real but bounded. ADDNet-2D beat XceptionNet by about 7 points on WildDeepfake and by about 11 points on DeepfakeDetection while matching it within 1 point elsewhere, and the authors write plainly that "the detection accuracy is only 76.25%" on their own set.

06Why are found deepfakes harder to detect?

Three properties of the set, all stated by the authors, explain the drop. The methods are unknown and mixed: wild fakes "are crafted by many different types, versions or even combinations of deepfake softwares", so no training set covers them. The footage is diverse: broadcasts, films, interviews and talks, several people in a scene, "different compression rates, resolutions and formats". And the fakes are good: they "were well-made, possibly owing to longer time of training with many high quality face images", so the obvious flaws that lab fakes carry are absent. A detector trained on lab fakes has learned the traces of one pipeline at one quality; the definition of a deepfake covers more pipelines than any one dataset holds.

07What are the limits of the dataset?

Four. The metric is accuracy, not AUC, so the published numbers cannot be placed directly in a table of frame-level AUC results. The fakes date from 2020 and earlier; the Deepfake-Eval-2024 benchmark repeated the exercise with 2024 fakes and found open-source video detectors' AUC fell by about 50 percent against their original benchmarks (Chandra et al., 2025), so WildDeepfake's gap is the 2020 reading of a problem that has not closed. Labels depend on uploaders' titles and annotators' judgement rather than ground truth about generation. And the paper gives no information on the people in the videos or on consent, which the researcher-made sets with volunteer actors do record. Within those limits, WildDeepfake remains the first published measurement of how far lab accuracy falls on fakes that actually circulated.