GAN versus diffusion faces: where the detection artefacts come from

This page is part of the deepfake detection hub and extends the generated faces page, which describes the StyleGAN family and the human studies on its output. Here the subject is the trace each generator leaves and what that means for a detector trained on the other.

01How does a GAN make a face, and where does the artefact come from?

A GAN generator maps a short random vector to a full image. Frank et al. describe the StyleGAN instance in their study as a mapping from 100 numbers to a 1,024 by 1,024 image, and note that "In typical GAN architectures, the latent vector gets successively upsampled until it reaches the final output dimension" (arXiv 2003.08685). Each upsampling step doubles the resolution by filling in pixels from their neighbours, and that filling leaves a regular pattern.

The pattern is easiest to see in the frequency domain. The authors apply a two-dimensional discrete cosine transform (DCT) to each image and average the spectra of 10,000 images; real FFHQ photographs follow the smooth fall-off expected of natural images, while StyleGAN images show "strong high frequencies components" and a "grid-like pattern scattered throughout". The paper's claim is structural: the artefacts "are consistent across different neural network architectures, data sets, and resolutions" and "are caused by upsampling operations found in all current GAN architectures". StyleGAN2's own authors had already found and removed one visible artefact, the water-droplet blob caused by instance normalisation (arXiv 1912.04958); the frequency artefact is a separate and more general trace.

02How well does the GAN artefact separate real from fake?

Very well, in the uncompressed case. Frank et al. train a ridge regression, the simplest linear classifier, on 10,000 real FFHQ images and 10,000 StyleGAN images. On raw pixels it reaches an accuracy of 75.78 percent on the test set; on DCT coefficients the same classifier reaches 100.00 percent, a gain of 24.22 points (Table 1). To test the cause, they retrain StyleGAN at 256 by 256 with three upsampling methods and repeat the experiment: the DCT classifier scores 98.24 percent against nearest-neighbour upsampling, 85.96 percent against bilinear and 84.20 percent against binomial (Table 2). Smoother upsampling weakens the trace; none of the three removes it.

Test accuracy in percent, ridge regression, from Frank et al. Tables 1 and 2. The cross-generator result from the same year points the same way: Wang et al. (CVPR 2020) report that a classifier trained on ProGAN alone "is able to generalize surprisingly well to unseen architectures, datasets, and training methods" across 11 CNN generators, with careful augmentation (arXiv 1912.11035). Within the GAN family, one generator's artefacts predict another's.

03How does a diffusion model make a face differently?

A diffusion model does not upsample a latent vector in one pass. Ho et al. define it as "a parameterized Markov chain trained using variational inference to produce samples matching the data after finite time", trained to reverse "a Markov chain that gradually adds noise to the data in the opposite direction of sampling until signal is destroyed", with T set to 1,000 steps in all their experiments (arXiv 2006.11239). The image emerges from repeated denoising at full resolution rather than from a stack of upsampling layers, so the mechanism Frank et al. identified is not present in the same form.

That does not make diffusion output trace-free. Corvi et al. estimate an artificial fingerprint from the noise residuals of 1,000 images per generator and look at its Fourier amplitude. "For all GANs, strong peaks are clearly visible in the spectra", and "the same happens with some recent diffusion models", GLIDE and Latent Diffusion among them, while "such peaks are much weaker" for ADM and one commercial text-to-image system (arXiv 2211.00680). The GAN trace is uniform across the family; the diffusion trace varies by model, and for some models is faint.

04What happens when a GAN detector meets diffusion images?

Corvi et al. test four detectors trained only on ProGAN against twelve generators, GAN and diffusion, with real images from COCO, ImageNet and UCID. Averaged over the twelve, the best detector, Grag2021, scores 75.8 percent accuracy and 92.8 AUC on uncompressed images. In the social-network condition, a random crop "resized to 200 × 200 pixels, and compressed using a random JPEG quality factor from 65 to 100", it falls to 61.5 percent accuracy at 90.8 AUC, and the spectrum-based detector falls from 70.5 percent and 75.2 AUC to 52.7 on both (Table 1). The authors' summary is that "the performance is acceptable in terms of AUC, but almost random in terms of accuracy": the ranking survives, the fixed threshold does not.

Averages over 12 test generators, percent, from Corvi et al. Table 1. Training on a diffusion model helps on its relatives: Grag2021 trained on Latent Diffusion alone averages 66.9 percent and 89.2 AUC, fusing it with the ProGAN model gives 64.5 percent and 94.1 AUC, and calibrating the fused model gives 87.8 percent at the same AUC (Table 2). The conclusion is blunt: "detectors trained only on GAN images work poorly on these new images", and "Including a DM model in training can help to detect images generated by similar diffusion models but results can be unsatisfactory for others". The generated-image detection benchmarks page lists the datasets built since to test exactly this.

05Why does compression hurt the frequency route most?

Both papers point at the same step. Frank et al. report that their frequency classifiers "are more robust to common image perturbations" than pixel ones in their tests, yet the artefact they rely on lives in the high frequencies. Corvi et al. advise detector builders to "avoid resizing, a common practice in deep learning to adapt images to fixed input layers, as this entails image resampling and interpolation, which may erase the subtle high-frequency traces". JPEG compression discards high-frequency DCT coefficients by design, so the same operation a platform applies to every upload removes the band where the upsampling trace sits. The ForgeryNet dataset built 36 perturbation types into its fakes for this reason, and the sibling page on whether Content Credentials survive upload shows the same re-encode breaking a cryptographic binding rather than a statistical one.

06What this means for detecting a face today

A face generated by a GAN in 2019 and a face generated by a diffusion model in 2024 are different objects to a detector. The first carries a uniform, structural trace that a linear model can read at 100 percent in clean conditions; the second carries a model-specific trace that is strong for some generators, weak for others, and in both cases degraded by the resizing and compression every shared image goes through. On fakes that circulated in 2024, three open-source image detectors scored 0.56, 0.53 and 0.52 AUC against 0.94 to 0.99 on their own benchmarks (arXiv 2503.02857). The deepfake detection tips tested against the evidence page shows that the viewer's eye does no better.

A detector trained on one generator family learns that family's trace; nothing in the published results says the trace carries across to a generator built on a different mechanism.