Frequency-domain deepfake detection: what F3Net, SPSL and SRM read in the spectrum

This page belongs to the deepfake detection hub and describes one of the three detector families compared there: the frequency family, which DeepfakeBench represents with F3Net, SPSL and SRM.

01Why does a generator leave a trace in the frequency domain?

Most face generators and face swappers build an image by starting small and upsampling, layer by layer, until the output reaches its final size. Frank et al. (ICML 2020) showed that in frequency space GAN-generated images "exhibit severe artifacts that can be easily identified", that these are consistent across architectures, datasets and resolutions, and that they "are caused by upsampling operations found in all current GAN architectures" (arXiv 2003.08685). Their method was a type-II discrete cosine transform averaged over 10,000 images; real photographs follow the usual fall-off of natural image statistics, while generated ones carry extra high-frequency energy and a grid-like pattern.

A face swap inherits the same problem. The swapped region is produced by a network that upsampled it, then blended into a frame that was not, so the spectrum of the face region differs from the spectrum of its surroundings. The SPSL authors put it in one sentence: "cumulative up-sampling will result in obvious changes in the frequency domain, especially in the phase spectrum" (arXiv 2103.01856). The page on artefacts in fully generated faces covers the same mechanism for images with no real face underneath.

02What do F3Net, SPSL and SRM each read?

The three detectors answer the same question with three different transforms. The table gives the one-line version; the paragraphs after it give the mechanism as each paper states it.

F3Net (Frequency in Face Forgery Network) takes "two different but complementary frequency-aware clues, 1) frequency-aware decomposed image components, and 2) local frequency statistics" and learns them in a two-stream framework; the authors "apply DCT as the applied frequency-domain transformation" and report the largest gains on low-quality, heavily compressed FaceForensics++ media (arXiv 2007.09355).

SPSL (Spatial-Phase Shallow Learning) keeps the pixels and adds the phase spectrum, on the argument that the phase "preserves abundant frequency components that provide extra information and complement the loss of the amplitude spectrum". It also makes the network shallower on purpose: "we reduce the receptive fields by shallowing the network to suppress high-level features and focus on the local region", because local texture mattered more than semantics in the authors' experiments (arXiv 2103.01856).

SRM starts from a failure: "current CNN-based detectors tend to overfit to method-specific color textures and thus fail to generalize". Image noise strips colour textures away, so the authors "propose to utilize the high-frequency noises for face forgery detection", extracting them at multiple scales and fusing them with RGB features through two attention modules (arXiv 2103.12376).

03How do the three score in DeepfakeBench?

DeepfakeBench re-implemented all three because none had public code at the time, trained each on FaceForensics++ c23 and scored every detector at frame level on six within-domain and eight cross-domain test sets (arXiv 2307.01426). The frequency rows of the DeepfakeBench table read, within-domain average then cross-domain average: F3Net 0.9449 and 0.7645, SPSL 0.9408 and 0.7875, SRM 0.9359 and 0.7760. The Xception baseline, trained the same way, scored 0.9450 and 0.7718.

Two things follow. Within the training distribution the frequency detectors gain nothing over the baseline; F3Net's 0.9449 and Xception's 0.9450 are the same number. On unseen sets SPSL and SRM pull ahead by about one to two points of AUC on average, and SPSL's advantage is concentrated on the two Celeb-DF sets, where it holds the best value in the whole table. The authors' appendix singles this out: SPSL "achieves an impressive average score of 78.75% in cross-domain evaluation", and they attribute the gain to the phase spectrum, which "can capture up-sampling artifacts present in many forgery processes".

04Where does the frequency signal fail?

The signal lives in high frequencies, and high frequencies are what compression removes. DeepfakeBench's augmentation study found that "compression and blur may significantly degrade the performance of SPSL in the DFD and DFDCP datasets, possibly due to their tendency to obscure high-frequency details". The same table shows every frequency detector losing more than ten points of AUC between FaceForensics++ c23 and c40: F3Net from 0.9635 to 0.8271, SPSL from 0.9610 to 0.8174, SRM from 0.9576 to 0.8114. The compression page sets those drops beside the spatial detectors' drops, which are of the same size.

Training recipe matters as much as the transform. DeepfakeBench's Table 10 reports SRM trained on FaceForensics++ c23 without and with the benchmark's "tricks" (augmentation and pre-training): on Celeb-DF v2 the AUC rose from 0.5993 to 0.7552, on DFD from 0.6029 to 0.8120, and on DFDCP from 0.5995 to 0.7408. A frequency detector trained plainly generalised no better than chance plus ten points; the same architecture with the common recipe reached the numbers in the main table.

05Why numbers differ between sources

The original papers report higher cross-dataset figures than DeepfakeBench does, and the reasons are protocol. The papers used their own preprocessing, frame counts, augmentation and sometimes video-level scoring; DeepfakeBench fixes all four for every detector, scores at frame level, and reproduced F3Net, SPSL and SRM from the papers' descriptions rather than from released code. DF40 (2024), which tests the same detectors on 40 newer forgery methods, adds a second caution: "SRM, SPSL, RECCE, and RFM, all state-of-the-art deepfake detectors using Xception as their backbone, achieve results similar to the simple baseline Xception" (arXiv 2406.13495). On the forgeries of 2020 the frequency signal bought a measurable margin; on the forgeries of 2024 that margin was not visible.

No single benchmark score predicts how a detector performs on unseen manipulations, and the frequency family is the clearest case: its best result and its collapse under compression come from the same table.