Face X-ray deepfake detection: the blending boundary and where it fails

This page is part of the deepfake detection hub. It belongs with the cross-dataset generalisation story, because Face X-ray was the first widely cited detector built to generalise to unseen manipulations, and its scores show both why the idea works and where it stops.

01What is a face X-ray?

The paper defines a face X-ray as "a greyscale image that reveals whether the input image can be decomposed into the blending of two images from different sources", showing "the blending boundary for a forged image and the absence of blending for a real image" (Li et al., 2020). The premise is that "most existing face manipulation methods share a common step: blending the altered face into an existing background image", and that the two sides of that boundary differ in sensor noise, compression history and colour statistics, however good the synthesised face is. A real photograph has no such seam, so its X-ray is all zeros.

The method therefore "only assumes the existence of a blending step and does not rely on any knowledge of the artifacts associated with a specific face manipulation technique". That is a different bet from a classifier such as Xception, which learns whatever separates the fakes it was shown from the real images it was shown and has no reason to prefer the boundary over a generator's own fingerprint.

02How is it trained without fakes?

Because every real image has an empty X-ray, training data can be manufactured from real images alone. The authors take two real faces with similar landmarks, cut a mask around one, deform the mask with a random piecewise affine warp and Gaussian blur so that its shape varies, colour-correct the foreground to the background, and paste it in. The blended image is a positive example and the mask boundary is its ground-truth X-ray. The paper calls this set BI, "the blended data samples composited using real images in FF++", and generates it on the fly during training.

The network is HRNet, with its four resolutions concatenated to 64 by 64, a 1 by 1 convolution and a bilinear upsampling to a 256 by 256 X-ray, trained for 200,000 iterations at batch size 32 with a 50,000 iteration warm start. A classification head on the predicted X-ray gives the real-or-fake score, weighted against the X-ray prediction loss by a factor of 100; the ablation shows a weight of 10, 100 or 1,000 gives 98.4, 98.5 and 98.6 percent AUC while no X-ray loss gives 94.5. The whole pipeline is a learned version of the question "is there a pasted region here?", which is also the question that separates face swaps from reenactment.

03What did the paper report?

The paper's Table 1 tests generalisation across the four FaceForensics++ manipulations. Xception trained on the DeepFakes subset scores 99.38 AUC on DeepFakes but 75.05 on Face2Face, 49.13 on FaceSwap and 80.39 on NeuralTextures. Face X-ray trained on DeepFakes plus blended images scores 99.12, 97.64, 98.00 and 97.77 on the same four, and trained on blended images alone, with no fakes at all, it scores 99.17, 98.57, 98.21 and 98.13 (Li et al., 2020, Table 1).

Table 2 moves to other datasets. Trained on FaceForensics++, Xception scores 87.86 AUC on the Google DFD set, 48.98 on the DFDC preview and 36.19 on the first Celeb-DF release of 408 real and 795 fake videos. Face X-ray trained on blended images alone scores 93.47, 71.15 and 74.76; with FaceForensics++ added, 95.40, 80.92 and 80.58. Table 7 tests other blending methods when training used alpha blending: 99.46 AUC on alpha blending, 94.62 on Poisson blending and 99.90 on deep blending.

All figures are frame-level AUC in percent from the paper's Table 2, with the paper's own Xception baseline.

04What does the Face X-ray deepfake detector score in DeepfakeBench?

Face X-ray's code was never released. DeepfakeBench's Table 1 marks it "Unpublished code, reproduced by us", one of five detectors the benchmark authors re-implemented "adhering to the instructions specified in the original papers" (Yan et al., 2023). Every score below is from that reproduction, trained on FaceForensics++ c23 with the benchmark's shared preprocessing and 32 frames per video, so it is the fairest comparison available but not the paper's own number.

Within FaceForensics++ the reproduction holds up: 0.9592 frame-level AUC on c23 and 0.9872 on the Face2Face subset, the best of 15 detectors in that column. Under heavy compression it falls to 0.7925 on c40, consistent with the paper's own finding that overall AUC drops from 87.35 percent on the lightly compressed release to 61.6 percent on the heavily compressed one. Across datasets the picture splits. Face X-ray takes the FaceShifter column at 0.6553, the only detector above 0.65 there, and scores 0.8989 on UADFV, 0.7655 on DFD, 0.7093 on Celeb-DF v1 and 0.6786 on Celeb-DF v2. But on DeeperForensics-1.0 it scores 0.5531, close to chance and the second lowest cross-dataset cell in the table after Meso4 on DFD, where FWA reaches 0.9334 and even Meso4 0.9113. Its cross-dataset average of 0.6985 ranks 14th of 15, above only Meso4 (published detector results by test set). The DeepfakeBench authors still single it out: "Face X-ray and DSP-FWA stand out as the exception, demonstrating exceptional performance in cross-manipulation detection", meaning across the FaceForensics++ methods, not across datasets.

A second reproduction gives video-level numbers. LipForensics, training on FaceForensics++ and averaging frame scores per video, reports Face X-ray at 79.5 percent AUC on Celeb-DF v2, 65.5 on DFDC, 92.8 on FaceShifter and 86.8 on DeeperForensics, an 81.2 average, against 75.3 for Xception (Haliassos et al., 2021, Table 2). The DeeperForensics figure there is 86.8, not 0.55, which the next section explains.

05Why does it fail on DeeperForensics-1.0?

Three causes are on the record. The first is in the paper's own limitations section: "our method relies on the existence of a blending step. Therefore, when an image is entirely synthetic, it is possible that our method may not work correctly." A face produced whole by a generator, or a swap whose output is blended so smoothly that the two sides match in noise and colour, leaves no seam to find. The second is compression and resolution, which the paper states as a drop from 87.35 to 61.6 percent AUC between the two FaceForensics++ releases; DeeperForensics-1.0 applies seven kinds of real-world perturbation to its test videos, and a boundary signal lives in exactly the high-frequency detail those remove. The third is the protocol. DeepfakeBench scores single frames; LipForensics averages over a video, which smooths out frames where the seam is invisible, and the two reproductions differ in preprocessing and training details. The gap between 0.5531 and 86.8 on the same dataset is a reminder that a detector's score depends on who ran it and how.

The authors also note that "one can develop adversarial samples to against our detector". A method that keys on one physical trace invites an attacker to remove that one trace, which later blending-based detectors address by varying the trace rather than by abandoning the idea.

06What the benchmark says to use it for

Face X-ray is the right tool when the question is "does this swap generalise across swapping methods?" and the fakes are reasonably clean: it holds between 0.9592 and 0.9872 within FaceForensics++ and leads on FaceShifter. It is the wrong tool for heavily compressed or perturbed video and for fully synthesised faces, where it is near chance. Like every detector score it is an inference from pixels, which a found watermark or a signed manifest outranks; none of the political deepfake records with an official file rested on a score. The DeepfakeBench harness behind these numbers is described on its own page, and the cross-dataset page sets Face X-ray's drop beside the drops of the other 14 detectors.