SPSL: phase spectrum deepfake detection with the leading cross-dataset average

This page is part of the deepfake detection hub and takes one of the three frequency-domain detectors explained there on its own, because it is the one that heads the cross-dataset column.

01What does SPSL read that a plain CNN does not?

The paper's starting observation is about generators rather than detectors: "up-sampling is a necessary step of most face forgery techniques, and cumulative up-sampling will result in obvious changes in the frequency domain, especially in the phase spectrum" (arXiv 2103.01856). A swapped face is produced by a network that enlarges a small representation step by step; each step leaves a trace in the Fourier transform of the result. Earlier frequency work read the amplitude spectrum. The SPSL authors argue that the phase spectrum "preserves abundant frequency components that provide extra information and complement the loss of the amplitude spectrum", and their first figure shows the average pixel difference of the phase spectrum rising sharply with each additional upsampling, measured over 1,000 images.

The detector therefore takes two inputs that describe one crop. The RGB image goes in as usual. The phase spectrum is computed with a discrete Fourier transform, converted back into a spatial image ("the IDFT of the absolute value of the pristine phase spectrum", in the paper's words) and stacked with it.

02Why is the network shallow?

The second idea is a claim about where the evidence sits. The authors "notice that local texture information is more crucial than high-level semantic information for the face forgery detection task", because the semantics of a face, eyes, nose, a mouth, are shared by real and forged images, while the forgery traces live in local texture. A deep network builds up large receptive fields and high-level features; SPSL removes most of it. The implementation keeps only Xception blocks 1 to 3 and block 12, dropping the middle of the network to "reduce the receptive fields by shallowing the network to suppress high-level features and focus on the local region".

The same cut applied to two ResNets gave the same direction of change, from 66.90 to 71.78 AUC on Celeb-DF for ResNet-34 and from 70.05 to 73.09 for ResNet-50, which the authors offer as evidence that SPSL "is a general framework for various backbones". The Xception as a deepfake detector page covers the full backbone that SPSL and most of its rivals start from.

03What did the ablation show?

Table 5 of the paper separates the two ideas. Four models were trained on FaceForensics++ at the high quality setting and tested on Celeb-DF, the cross-dataset condition the paper is built around.

Each half helps on its own and the two add: "When combining phase spectrum and shallow network, SPSL gets the best performance and the AUC score increase by about 13%." Within the training distribution SPSL gains little. Its Table 1 puts it at 95.32 AUC against Xception's 94.86 on FaceForensics++ high quality, and 82.82 against 81.76 at low quality, which the abstract summarises as "comparable results on single dataset evaluation".

04How does SPSL score in its own paper?

The paper's headline cross-dataset table, Table 3, compares AUC on Celeb-DF for detectors trained on FaceForensics++, taking most rival figures from the Celeb-DF paper. SPSL reaches 76.88 on Celeb-DF while scoring 96.91 on the FaceForensics++ DeepFakes class; the runner-up, Two-branch, scores 73.40 on Celeb-DF, F3-Net 65.17, Xception trained at c40 65.50, DSP-FWA 64.60, Capsule 57.50 and Meso4 54.80. The Celeb-DF page describes the set behind the column.

The paper also runs a five-way classification, real plus the four FaceForensics++ manipulations, and reports recall per class. On raw video SPSL recovers 99.05 percent of DeepFakes, 97.20 of Face2Face, 97.63 of FaceSwap and 91.40 of NeuralTextures against 97.84, 96.68, 96.84 and 87.67 for Xception, with NeuralTextures the hardest class for both, as it is in every benchmark on this site.

05How does SPSL score in DeepfakeBench?

DeepfakeBench re-implemented SPSL, because its code was not public, and trained it under the same settings as fourteen other detectors on FaceForensics++ c23 (arXiv 2307.01426). In Table 3 its within-domain average is 0.9408 frame-level AUC with no column wins, and its cross-domain average is 0.7875, the highest of the fifteen, with wins on both Celeb-DF columns.

The authors single the result out in their appendix, where they "highlight the SPSL detector, which achieves an impressive average score of 78.75% in cross-domain evaluation" and credit the phase spectrum's ability to "capture up-sampling artifacts present in many forgery processes". The cross-dataset scores by model carry every cell of the column with its conditions, and the cross-dataset generalisation page explains why a 0.19 gap between within-domain and cross-domain averages is ordinary for this table.

06Where does the phase signal fail?

Two findings qualify the headline. DeepfakeBench's augmentation study, which used SPSL as its frequency example, found that "compression and blur may significantly degrade the performance of SPSL in the DFD and DFDCP datasets, possibly due to their tendency to obscure high-frequency details"; the page on the DFDC Preview set lists the full DFDCP column where that shows. A detector that reads upsampling traces reads less when a platform's re-encoding has smeared them. The same study found that removing all augmentations improved within-domain performance by about 2 percent for both Xception and SPSL.

The second finding is from DF40 (2024), which trained SPSL and three other Xception-based detectors on forty newer generators and reported that they "achieve results similar to the simple baseline Xception" (arXiv 2406.13495). The SPSL authors had already written the limitation into their own conclusion: the method "suffers from a transferability drop when encountering an entirely different type of face forgery manipulation", since a model trained on identity swaps may miss expression swaps. The best cross-dataset average in a 2023 table is a statement about the 2023 test sets. It is also a statement about one layer: the deepfake detection and prevention layers page places a detector score beside the capture and provenance checks that do not have to infer anything.

07Why numbers differ between sources

SPSL's Celeb-DF score is 76.88 AUC in its own paper and 0.7650 on Celeb-DF v2 in DeepfakeBench. The two runs differ in implementation (the authors' code against DeepfakeBench's re-implementation), in preprocessing and frame sampling, in training quality (the paper trains at c23 for the ablation and reports Table 3 without a stated compression; DeepfakeBench fixes c23 for everyone), and in the Celeb-DF version and test list. This site quotes the DeepfakeBench figure wherever a comparison across detectors is made, because every row in that table was produced under one protocol, and quotes the paper's figure only as the authors' own result.