Compression and deepfake detection: the numbers from FF++ and the wild

This page belongs to the deepfake detection hub and isolates one variable that the cross-dataset generalisation page folds into a larger drop: the fakes, the generator and the source footage stay fixed here, and only the encoder setting changes.

01What c23 and c40 mean

FaceForensics++ was released at three quality levels. The raw videos are the manipulated frames as the generators produced them. The paper then compresses every video with H.264, "which is widely used by social networks or video-sharing websites", at two settings: a constant rate quantisation parameter of 23, which the authors call high quality and describe as "visually nearly lossless", and a quantisation of 40, which they call low quality (arXiv 1901.08971). The dataset releases label these HQ and LQ as c23 and c40.

A higher quantisation parameter discards more of the fine detail in each frame, and the detail it discards first is the high-frequency texture that most face forgery detectors have learned to look for. That is why compression is not just another perturbation: it removes the evidence rather than adding noise around it. DeepfakeBench, the field's reference benchmark, uses the c23 release of FaceForensics++ by default, so most published scores describe the lightly compressed condition.

02How much does compression cost a detector on the same fakes?

The FaceForensics++ paper answers this with one table. Seven detection methods were trained and tested on all four manipulations at each quality level, and binary accuracy was recorded at raw, HQ and LQ.

Accuracy in percent, from Table 2 of arXiv 1901.08971, models trained on all four manipulations. Two patterns hold across the rows. Every method loses most of its margin between HQ and LQ, not between raw and HQ: the hand-crafted steganalysis features drop 27 points at the first step and another 15 at the second, while XceptionNet loses 3.5 and then 14.7. And the gap between methods widens as quality falls: at raw the six face-crop methods sit within about four points of each other, at LQ they span 25. The authors' reading is that "domain-specific information in combination with a XceptionNet classifier shows the best performance in each test". Quality 40 is the level at which a strong 2019 detector gets one in five decisions wrong on fakes it was trained on.

03Does training on c23 survive a c40 test?

DeepfakeBench (NeurIPS 2023) asks a different question: what happens when a detector trained at one quality is tested at another. Its Table 3 trains all fifteen detectors on FaceForensics++ c23 and reports frame-level AUC on the c23 and c40 test splits of the same videos, among other sets (arXiv 2307.01426). Xception falls from 0.9637 to 0.8261. UCF, the best within-domain detector in the table, falls from 0.9705 to 0.8399. SPSL, a frequency-domain method and the best cross-dataset detector, falls from 0.9610 to 0.8174, and Face X-ray from 0.9592 to 0.7925. Meso4, the weakest, falls least in absolute terms, from 0.6077 to 0.5920, because it had the least to lose.

Every one of the fifteen rows scores lower on c40. The thirteen detectors above 0.84 AUC on c23 each lose between 0.06 and 0.17 on the c40 column, and the two MesoNet detectors lose 0.02 and 0.03 because they had little to lose. The full table, with every test set and the cross-dataset averages, is on the frame-level AUC table page; the c40 drop there is the only column where the training and test videos are identical and only the encoder differs, which makes it the cleanest measure of compression alone in the published literature.

04Why does compression remove the evidence?

Most learned detectors key on traces left by the generator's final layers: upsampling patterns, blending seams, texture inconsistencies between the swapped face and the surrounding skin. These traces live in the high-frequency band of the image, and a lossy codec is designed to throw that band away first because viewers notice it least. The DeepfakeBench authors observe the mechanism in their augmentation study: for the compressed c40 data, Gaussian blur during training "demonstrate[s] effectiveness in both Xception and SPSL detectors, as they simulate the effects of compression on the data during training". A detector that has seen blurred faces has learned not to depend on the detail compression will remove.

The same study shows the cost. For cross-dataset tests the authors report that compression and blur augmentations "may significantly degrade the performance of SPSL in the DFD and DFDCP datasets, possibly due to their tendency to obscure high-frequency details". A frequency-domain detector is a detector of exactly the information compression destroys, so hardening it against compression blunts its main instrument. There is no setting that is free.

05What do people lose to compression?

Compression hurts human viewers less than it hurts detectors, which is part of why fakes circulate. The FaceForensics++ user study put 204 participants in front of pristine and manipulated images for two to six seconds each, and recorded an average accuracy of 68.69 percent on raw frames, 66.57 percent on high quality and 58.73 percent on low quality. People lost ten points between raw and c40; XceptionNet lost eighteen. On the paper's own numbers the detector still beats the viewer at every quality level, but its advantage shrinks from 31 points at raw to 22 at c40.

06What happens in the wild, where the compression is unknown?

On a curated set the compression is a known constant, chosen once by the dataset's authors. On a social platform it is unknown and compounded: a clip is encoded by the creator's software, re-encoded on upload, often screen-recorded and uploaded again. Deepfake-Eval-2024 collected fakes that circulated on 88 websites in 2024 and found that open-source detectors' AUC fell by about 50 percent for video, 48 percent for audio and 45 percent for images compared with their original benchmarks (arXiv 2503.02857); the best off-the-shelf open-source video model scored 0.63.

Compression is one of the causes stacked inside that number, alongside unseen generators and a different kind of real footage, and the in-the-wild studies cannot separate it because the encoding history of a found clip is unrecoverable. The FaceForensics++ and DeepfakeBench tables are the controlled measurements that say how large the compression share can be on its own: eighteen points of accuracy at quality 40 for the strongest 2019 detector, and 0.06 to 0.17 AUC for the thirteen strongest detectors in the 2023 benchmark.

07How to read a published score with compression in mind

A detector score without its compression level is incomplete in the same way a face recognition error rate is incomplete without its threshold. Three habits follow from the tables above. Ask which release the test used: a FaceForensics++ figure at raw and one at c40 can differ by eighteen points for the same model. Treat c23 as the lab condition and c40 as the closest proxy for a platform re-encode. And when a paper reports only one level, assume it is the easiest one unless it says otherwise.

The pattern is not specific to faces. The audio challenge series tested its detectors on codec-compressed speech from 2021 and found that solutions "offer some resilience to compression effects" but fail across source datasets; the ASVspoof editions page gives the figures. In liveness testing, the face anti-spoofing protocols hold out capture devices rather than codecs, but for the same reason: the sensor path changes the evidence. And in a deployed pipeline, compression is one of several steps between the camera and the score where accuracy leaks away; the face recognition system components page maps them.