Generalizable deepfake detection: what has been tried, and the scores

This page belongs to the deepfake detection hub and extends the cross-dataset generalisation page, which measures the gap. Here the question is what authors did about it, and what their own tables report.

01What does generalization mean for a deepfake detector?

A detector generalizes when its score holds on forgeries made by methods, people, scenes or compression it did not see in training. The standard test trains on one dataset, usually FaceForensics++, and scores on others. DeepfakeBench describes the field's earlier problem as a lack of uniform data processing, experimental settings and metrics, which made comparisons unfair (arXiv 2307.01426).

DF40's authors argue the protocol itself hides the problem. Their abstract says that training on FF++ and testing on other prevalent sets has been treated as a "golden compass", and that FF++ "contains out-of-date forgery techniques from the past four years" (arXiv 2406.13495). A score on an old test set can therefore say little about a new generator.

02Does stronger augmentation and pre-training close the gap?

It narrows the gap for every detector, and by roughly the same amount. DeepfakeBench's Table 10 compares three detectors trained on FF++ c23, with and without the data augmentation and pre-training its framework applies. Xception rose from 0.5175 to 0.7365 AUC on Celeb-DF-v2, RECCE from 0.6083 to 0.7319 and SRM from 0.5993 to 0.7552, while their in-dataset FF++ c23 scores fell slightly, for example Xception from 0.9893 to 0.9637.

The same paper reports that gains from augmentation are not universal. Removing all augmentation improved in-dataset AUC by about 2 percent for both Xception and SPSL, and compression and blur "may significantly degrade the performance of SPSL" on DFD and DFDCP, possibly because they obscure high-frequency detail. An earlier image-detection study found a ProGAN-trained classifier generalized to unseen generators "with careful pre- and post-processing and data augmentation" (Wang et al., CVPR 2020). The recipe matters as much as the model.

03Do blended training fakes generalize better?

Two methods build their own fakes instead of relying on a dataset's. Face X-ray (CVPR 2020) trains a network to find the blending boundary, a step the authors say most manipulation methods share, and can be trained without fakes from any manipulation method (arXiv 1912.13458). Self-blended images (SBI, CVPR 2022) blend pseudo source and target images made from a single real image (arXiv 2204.08376); the SBI page and the Face X-ray page describe each in full.

The first three rows are the authors' own tables; the last is a third party re-implementation. The gains are large against a plain classifier on the authors' sets and shrink or vanish in the benchmark re-run, where Face X-ray's cross-dataset average of 0.6985 is below Xception's 0.7718.

04Do frequency cues help?

Frequency methods search for traces that generators leave in the spectrum rather than in pixels. For GAN images, one study reported "severe artifacts" in frequency space caused by the upsampling operations in all current GAN architectures (arXiv 2003.08685). The frequency-domain detection page covers the family.

In DeepfakeBench's Table 3, SPSL has the best cross-dataset average of the 15 detectors at 0.7875, against 0.7718 for Xception and 0.7801 for UCF, and the best scores on both Celeb-DF releases, 0.8150 and 0.7650. Its within-dataset average is 0.9408. That is a gain of about 0.02 over the plain baseline and a gap of about 0.15 to its own in-dataset score.

05Do foundation-model features help?

This is the newest attempt. UnivFD (2023) trains no network at all: it classifies fake versus real from the frozen features of a vision-language model, CLIP, using nearest neighbour or a linear probe (arXiv 2302.10174). The best conventionally trained network scored 94.19 mAP on generators from its own GAN family and 75.51 on unseen diffusion and autoregressive models; the linear probe scored 93.38 mAP over all 19 test generators, against 83.58 and 81.52 for two trained baselines. These are average precision scores on whole-image generators, not face-swap AUC, and are not comparable to the tables above.

DF40's third finding points the same way for face forgeries: "CLIP excels in deepfake detection than other baselines (such as Xception)", outperforming other detectors "across all scenarios". Its second finding is less encouraging for purpose-built designs: SRM, SPSL, RECCE and RFM "achieve results similar to the simple baseline Xception" across the new forgeries.

06What is still unsolved?

Every method above is scored on curated sets. On fakes that circulated in 2024, open-source detectors fell by about 50 percent in AUC for video, and the best off-the-shelf video model, GenConViT, went from 0.96 on its original benchmark to 0.63; the in-the-wild page gives the full table. The caveat for every row on this page is DF40's: a detector can lead on the datasets it is usually tested on and still, in DF40's words, "may not have obvious advantages over the baseline" on new forgeries.

Numbers differ between sources for three reasons: the test set's release (Celeb-DF has two), the training data (a method may add its own fakes) and the aggregation (frame-level or video-level). The published results page keeps each row with its conditions. Audio detection faces the same split between lab sets and found recordings, covered in the audio deepfake datasets guide.