Deepfake detection surveys: what the main review papers agree on

This page belongs to the deepfake detection hub. It reads three review papers side by side and traces their figures back to the benchmark tables, which are kept on the published benchmark scores page.

01Which surveys does this page read?

Three review papers, chosen because each has a detection section with numbers and each is a primary document rather than a commentary on one. They cover different periods, so they cannot be read as a count of how many agree.

02What do the surveys agree on?

They agree on one point above the others: a detector that scores well on the data it was trained for is not shown to work on data it has not seen. Each survey says so in its own words.

Mirsky and Lee observe that "the majority deepfake detection algorithms assume a static game with the adversary: They are either focused on identifying a specific artifact, or do not generalize well to new distributions and unseen attacks" (arXiv 2004.11138). Tolosana et al. write that manipulations "seem easy to be detected under controlled scenarios, i.e., when fake detectors are evaluated in the same conditions they are trained for", and that this motivates "further research on the generalisation ability of the fake detectors against unseen conditions" (arXiv 2001.00179).

Pei et al. put the same point as a design limit: most detection models target "common operational steps of a specific forgery method", and "this manner limits the model's generalization capabilities" (arXiv 2403.17881). The cross-dataset page shows how large the drop is for fifteen detectors in one benchmark.

03Which numbers do the surveys cite, and where do they come from?

The scores in surveys are copied from original papers and benchmark tables, so each can be traced. Mirsky and Lee state that the three most recent benchmark datasets, DFD, DFDC and Celeb-DF, "were released within one month of each other at the end of 2019", and that detectors "only achieved an AUC of 0.86, 0.76, and 0.66 on each of them respectively". The survey does not name the detector or the averaging. The Celeb-DF paper's Table 4 gives frame-level AUC for 13 methods on those sets; the best value in each of the three columns is 85.9, 75.5 and 65.5 percent, which round to the survey's three figures (arXiv 1909.12962).

Tolosana et al. report that fake detectors reach AUC "close to 100%" on first-generation databases such as UADFV and FaceForensics++ but degrade on newer ones, "in particular for the Celeb-DF database with AUC results below 60% in most cases". The same Celeb-DF table agrees: 10 of its 13 methods score below 60 percent on Celeb-DF (a count made from the table, not a figure the survey states). The two Celeb-DF releases differ in size, as the page on Celeb-DF versions explains.

Pei et al. benchmark forgery detection on FaceForensics++ and four cross-dataset targets, and their Table 15 carries figures that match the detectors' own papers. It lists Zheng et al., the FTCN authors, at 86.90 on Celeb-DF v2, 74.00 on the Deepfake Detection Challenge set and 98.80 on DeeperForensics-1.0; the FTCN paper's Table 4 reports 86.9, 74.0 and 98.8 for the same sets as video-level AUC (arXiv 2108.06693).

04Why do survey figures differ from benchmark tables?

Survey figures and benchmark figures describe different protocols, so they should not be set side by side without checking four conditions.

• Training set. Pei et al.'s Table 15 mixes methods trained on FaceForensics++, its real subset, its high and low quality releases, self-built data and other sets, as its caption notation shows.
• Level of scoring. The FTCN figures are video-level; the DeepfakeBench table is frame-level, so the same detector can have two numbers (arXiv 2307.01426).
• Version of the dataset. Mirsky and Lee's 0.66 on Celeb-DF was written about the 2019 benchmark. DeepfakeBench gives Xception 0.7365 on Celeb-DF v2 after its own training run, a different measurement.
• Date. A 2020 survey could not include the 2025 in-the-wild results; the in-the-wild benchmark page covers those.

05What do the surveys leave open?

They leave open how to compare methods fairly. Pei et al. note for some generation tasks that results come "as presented in the original publications", with no fixed splits, so "quantitative fair comparisons are not possible". For detection they fix a self-dataset and four cross-dataset sets and report AUC and accuracy, which is closer to a protocol but still collects results from different training choices. Mirsky and Lee add a limit of a different kind: a detector built around a specific artifact assumes an attacker who does not adapt.

A survey is therefore best used as an index to the primary papers and datasets, with the numbers re-read in the source table. The reading list of detection papers on this site is organised that way.