Dataset bias in deepfake detection: what detectors actually learn

This page sits in the deepfake detection hub between the mechanism and the evidence. The cross-dataset drop explained page gives the drops model by model; this page asks what the detector was learning in the first place, and why the drop is the expected result rather than a surprise.

01What does dataset bias mean for a detector?

A detection training set has three properties a detector can latch onto that have nothing to do with the question "is this face real" (the what a deepfake video is page defines the alteration itself). The first is the generator: FaceForensics++ was made with four methods, DeepFakes, Face2Face, FaceSwap and NeuralTextures, each leaving its own pattern of artefacts (arXiv 1901.08971). The second is the compression: the set is released as raw video and at two H.264 settings, constant rate factor 23 and 40, which the paper calls HQ and LQ. The third is the footage: 1,000 source videos taken from 977 YouTube clips, so the real and the fake halves share lighting, resolution and camera habits that another set will not.

A classifier trained with a real-or-fake label is free to use any of the three. Dataset bias is the name for the result: a detector whose score tracks "made by one of these four methods at this compression on this footage" rather than "manipulated".

02What does a detector trained on one manipulation learn?

The Face X-ray paper (CVPR 2020) ran the cleanest version of the experiment. It trained Xception on the DeepFakes manipulation of FaceForensics++ alone and tested it on all four. The classifier scored 99.38 AUC on DeepFakes, the method it had seen, then 75.05 on Face2Face, 80.39 on NeuralTextures and 49.13 on FaceSwap, which is chance (arXiv 1912.13458, Table 1). Nothing about the task had changed; every test video was a manipulated face. The detector had learned DeepFakes, not manipulation.

The same paper trained its own method on DeepFakes plus blended real images, with no FaceSwap, Face2Face or NeuralTextures video at all, and scored 99.27, 98.43, 97.85 and 99.27 on the four. The difference is what the two models were asked to find. Xception was asked to separate two piles of frames and found the generator's signature. Face X-ray was asked to locate a blending boundary, "a greyscale image that reveals whether the input image can be decomposed into the blending of two images from different sources", and because every swap method of that era blended a face into a background, the boundary was present in all four.

03How does the cross-dataset grid measure it?

DeepfakeBench made the measurement systematic. Every one of its 15 detectors is trained on FaceForensics++ c23 with one preprocessing pipeline, 32 frames per video, and tested on eight other sets (arXiv 2307.01426, Table 3). The table is a grid of frame-level AUC values, and reading one row across it is reading how much of a detector's score came from the training set.

Frame-level AUC, DeepfakeBench Table 3, 2023. The published cross-dataset grid carries every cell for all 15 detectors with its conditions.

Three things in the row for Xception are the bias made visible. The c40 column is the same fakes at heavier compression, and the score falls from 0.9637 to 0.8261 with no change in generator. The Celeb-DF v2 and DFDC columns are different swap methods on different footage, and the score falls to 0.7365 and 0.7077. FaceShifter is a later swap method applied to the same 1,000 FaceForensics++ originals, and Xception scores 0.6249 on it, the lowest cell in its row: same footage, same compression, unseen generator. The best cross-dataset average in the table is SPSL at 0.7875, against within-dataset averages of 0.9450 for Xception and 0.9527 for UCF, so no detector in the benchmark escapes the pattern.

04Why is compression a dataset trait as well?

Compression leaves traces of its own, and a detector trained at one setting learns them. The FaceForensics++ paper reports XceptionNet's binary accuracy, trained on all four methods, as 99.26 percent on raw video, 95.73 percent at constant rate factor 23 and 81.00 percent at 40 (arXiv 1901.08971, Table 1). The fakes are the same; only the codec setting moved. DeepfakeBench trains on c23 only, which is why its c40 column is a cross-compression test as much as its Celeb-DF column is a cross-generator test; the three release settings are the same fakes at three codec qualities.

A platform re-encodes every upload, so a detector that scored on c23 meets something closer to c40 or worse in use. That is one of the measured causes behind the benchmarks built from circulating fakes, where open-source video detectors' AUC fell by about 50 percent in Deepfake-Eval-2024.

05Why do blending-based detectors generalise better?

Because they are trained to find something the dataset did not choose. Face X-ray's authors state the design rule plainly: most face manipulation methods "share a common step: blending the altered face into an existing background image", and the detector "only assumes the existence of a blending step". Self-blended images (SBI, CVPR 2022) push the idea one step further by making the training fakes from single real images, "reproducing common forgery artifacts (e.g., blending boundaries and statistical inconsistencies between source and target images)" so that the classifier learns "without overfitting to manipulation-specific artifacts" (arXiv 2204.08376). The paper reports gains of 4.90 and 11.78 points over the baseline on DFDC and DFDC Preview in cross-dataset evaluation.

The DeepfakeBench authors saw the same split in their cross-manipulation experiment, where Face X-ray and DSP-FWA "stand out as the exception, demonstrating exceptional performance in cross-manipulation detection". The gain is not free. Face X-ray's own row in the cross-dataset grid averages 0.6985, below Xception's 0.7718, with 0.5531 on DeeperForensics-1.0, whose fakes are made without the blending step the detector assumes.

06Does a bigger or newer dataset remove the bias?

It moves it. DF40 (2024) was built with 40 generation techniques because, in its authors' words, FaceForensics++ "contains out-of-date forgery techniques from the past four years", and its second finding is that "existing well-designed (SoTA) detectors may not have obvious advantages over the baseline" on new forgery types (arXiv 2406.13495). A set that covers more generators teaches a detector more signatures; it does not teach it the signature of next year's generator.

The footage carries a bias too. Celeb-DF v2 has 59 subjects; FaceForensics++ has 977 source clips; the Deepfake Detection Challenge recorded 3,426 paid actors. A detector learns the real side of its set as well as the fake side, and the dataset counts and sources table shows how differently each set sampled its people.

07How to read a published score with the bias in mind

A within-dataset AUC says how well a detector learned its training set. A cross-dataset AUC says how much of that learning was about manipulation rather than about the set, and it is the number to quote. When a paper reports only the first, the second is unknown, not high. A fair test of a detector on new material follows the steps on the evaluating a detector on your own data page: held-out footage, a fixed operating point and at least one unseen generator. Human judgement is not the control either: the synthetic faces and trust page shows raters trusting AI faces more than matched real ones. No single benchmark score predicts how a detector performs on unseen manipulations; the grid exists because the single score cannot.