Deepfake detection techniques compared: spatial, frequency, video

This page belongs to the deepfake detection hub and sits next to how deepfake detection works, which follows one clip from frames to a score. Here the question is narrower: when the data, the backbone and the training recipe are held constant, what does each family of cue buy?

01What are the three families of deepfake detection technique?

DeepfakeBench sorts its fifteen detectors into three model types in its Table 1. A naive detector "employs CNNs to directly distinguish deepfake content from authentic data": an image classifier such as Xception or EfficientNet-B4 is given the face crop and a real or fake label and left to find its own cues. A spatial detector adds a designed representation in pixel space, which the paper lists as "forgery region location, capsule network, disentanglement learning, image reconstruction, erasing technology"; UCF, Face X-ray, RECCE and FFD are the examples in the table. A frequency detector transforms the crop and looks for traces in the spectrum: F3Net, SPSL and SRM, which the paper describes as using "phase spectrum analysis and high-frequency noises" (arXiv 2307.01426).

The families share more than they differ on. Eight of the fifteen detectors run on an Xception backbone; EfficientNet-B4 uses EfficientNet, CNN-Aug a ResNet, Face X-ray HRNet, and the two Meso models, Capsule and RECCE their own designs. All fifteen are trained on the same FaceForensics++ training videos at compression level c23, with 32 frames sampled per video, and scored by frame-level AUC. That is what makes the comparison below meaningful: the only thing that varies is the cue.

02Which family scores best on the data it was trained on?

Within FaceForensics++ the differences are small. The paper's Table 3 gives each detector an average over six within-domain test sets: the c23 and c40 releases and the four manipulation methods. UCF leads at 0.9527, followed by Xception at 0.9450, F3Net at 0.9449, FFD at 0.9434 and CORE at 0.9431. SPSL, the frequency detector that wins across datasets, sits at 0.9408, and the plain EfficientNet-B4 at 0.9389. The five weakest within-domain scores belong to the shallow or older designs: Meso4 at 0.6097, MesoInception at 0.7571, Capsule at 0.8217, CNN-Aug at 0.8419 and FWA at 0.8549.

The authors draw one conclusion from this block that matters for every comparison page. Naive detectors, "which essentially rely on a straightforward CNN classifier, register high AUC values that are comparable to more sophisticated algorithms". The gap between a purpose-built cue and a plain classifier, measured on the training distribution, is about one hundredth of AUC.

All values are frame-level AUC for detectors trained on FaceForensics++ c23, from DeepfakeBench Table 3 (2023). The frame-level AUC table on this site carries every row with its test set.

03Which family generalises best to fakes it has not seen?

Across the eight cross-domain test sets (Celeb-DF v1 and v2, DeeperForensics-1.0, DFD, DFDC, DFDC Preview, FaceShifter and UADFV) the order changes and every number falls. SPSL has the best average at 0.7875, with SRM at 0.7760 and UCF at 0.7801 close behind. The naive Xception and EfficientNet-B4 tie at 0.7718, ahead of six of the ten purpose-built detectors. Face X-ray, a spatial detector built to find blending boundaries, has the lowest cross-dataset average in the table at 0.6985, pulled down by 0.5531 on DeeperForensics-1.0.

The column winners are spread across families. SPSL takes both Celeb-DF columns (0.8150 and 0.7650); UCF takes DFDC (0.7191), DFDC Preview (0.7594) and UADFV (0.9528); FWA takes DeeperForensics-1.0 (0.9334); Xception takes DFD (0.8163); Face X-ray takes FaceShifter (0.6553). No family wins everywhere, and the frequency family's lead on average is 0.0074 over the best spatial detector and 0.0157 over the best naive one. The cross-dataset generalisation page in this section gives the drops detector by detector.

04What does each cue family need from the data?

The families fail in different ways, and the DeepfakeBench ablations show why. Frequency cues are fragile under processing that smooths the spectrum: in the augmentation study, training with compression and blur "may significantly degrade the performance of SPSL in the DFD and DFDCP datasets, possibly due to their tendency to obscure high-frequency details". The same operations help on the heavily compressed c40 release, because they simulate compression during training. A frequency detector therefore needs its training compression to resemble its deployment compression, more than a spatial one does.

Spatial cues depend on the forgery leaving a visible seam or region. Face X-ray scores 0.9872 on the Face2Face split of FaceForensics++, the best of any detector on that column, and 0.5531 on DeeperForensics-1.0, whose fakes were made with a different swapping framework and heavy perturbation. A detector built for one kind of boundary does not transfer to a fake that has no boundary of that kind.

Naive cues depend most on the recipe around the network. Table 10 of the paper trains Xception twice: without pre-training and augmentation it scores 0.9893 within FaceForensics++ and 0.5175 on Celeb-DF v2, 0.5870 on DFD and 0.4894 on DFDC Preview, at or below chance; with them it scores 0.9637 within and 0.7365, 0.8163 and 0.7374 across. The same holds for RECCE and SRM. Much of what looks like a difference between techniques in the literature is a difference in these "tricks", which is why the paper holds them constant.

05Where do video detectors fit?

The three families above score frames. DeepfakeBench "currently adopts the frame level evaluation to build a fair basis for comparison among detectors", samples 32 frames per video, and lists video-level detectors as future work. A video detector adds a fourth kind of cue: inconsistency across time, such as a mouth that does not move with the speech or a texture that flickers between frames. DF40 (2024) reports a video-level model, I3D, scored by video-level AUC, alongside frame-level detectors on the same forty techniques. The deepfake videos page explains what changes when the unit of scoring is a clip rather than a frame.

The in-the-wild results give the only video-level comparison on circulating fakes. On Deepfake-Eval-2024, the open-source video model GenConViT scored 0.63 AUC off the shelf, and FTCN and Styleflow 0.50 and 0.51, at chance (Chandra et al., Table 3, 2025). None of the three families has a published score on that set, so the question of which cue survives platform re-encoding best is open.

06How to read a technique comparison

Three conditions decide whether two technique scores can be compared: the training set and compression, the test set, and the evaluation unit. A paper that trains on raw FaceForensics++ and reports video-level accuracy cannot be set beside one that trains on c23 and reports frame-level AUC, even if both call their method a frequency detector. The caveat holds across the field: no single benchmark score predicts how a detector performs on unseen manipulations, and the family label predicts even less than the score does.

The practical reading of DeepfakeBench Table 3 is that cue family is a second-order effect. The first-order effects are the training data, the backbone, pre-training and augmentation, and the distance between the training fakes and the test fakes. Readers who want the human side of this question, the cues that people rather than models can see, will find them on the signs of a deepfake page; the generators that produce the artefacts each family looks for are described under deepfake technology.

The frequency family (F3Net, SPSL, SRM) and its published cross-dataset scores are compared on the frequency-domain deepfake detection page.