Signs of a deepfake: what the studies say people can and cannot see

This page is part of the deepfake detection hub and is the video companion to can people spot AI faces, which covers synthetic still faces. Every sign below is paired with the study that measured how often people actually saw it.

01What counts as a sign of a deepfake?

A sign, in the sense the studies measure, is any property of the frame or clip that moves a viewer's judgement toward "fake". The FaceForensics++ authors describe the ones their face swaps leave as "a strong semantic change", a replaced face that does not quite belong to the head and lighting around it, and the ones their reenactments leave as "only subtle visual artifacts" (arXiv 1901.08971). Groh et al. (PNAS 2022) hand-labelled seven clip-level properties for their DFDC videos: graininess, blurriness, darkness, a flickering face, two people in frame, a floating distraction and the presence of a person with dark skin, and measured how each changed accuracy.

Two things follow. A sign is specific to a generator and a year: the artefact a 2017 autoencoder swap leaves at the jawline is not the artefact a 2024 diffusion model leaves, and generator papers remove known artefacts between versions. And a sign is only as useful as the share of fakes that show it, which is what the numbers below record.

02What did the FaceForensics++ user study measure?

The study showed 204 participants, mostly computer science students, single frames from the test set, half pristine and half manipulated by one of the four methods, at one of the three compression levels. Each frame was shown for a random limit of 2, 4 or 6 seconds "to mimic scenario where a user only spends a limited amount of time per image as is common on social media", then hidden before the question was asked. Each participant saw 60 frames, for 12,240 decisions in all.

Average accuracy was 68.69 percent on raw frames, 66.57 percent at the c23 compression level and 58.73 percent at c40. The paper's Table 7 breaks those averages down by manipulation, and the average hides a split between fakes people see and fakes they do not.

03Which manipulations can people see, and which can they not?

The two face swaps were visible. Participants identified DeepFakes frames as fake 77.60 percent of the time on raw frames and 73.18 percent at c40, and FaceSwap frames 76.12 and 64.26 percent. Those methods replace the whole face, so the sign is semantic: the face does not sit right on the head, or the skin tone and lighting disagree with the neck and background. Compression costs FaceSwap twelve points and DeepFakes four.

The two reenactments were not. Face2Face frames were called fake 49.60 percent of the time on raw frames and 43.86 at c40, chance or worse; NeuralTextures frames 32.28 percent raw and 39.07 at c40, well below chance, meaning participants judged most of them real. The authors' explanation is that these methods "do not introduce a strong semantic change", because the target keeps their own face and only the expression, or only the mouth region for NeuralTextures, is redrawn. Pristine frames were judged real 78.19 percent of the time when raw and 62.06 percent at c40, so compression also made real footage look fake to a third of viewers.

Percent of frames judged correctly, FaceForensics++ Table 7 (2019).

04Which cues helped in whole-video studies?

Groh et al. showed whole clips from the DFDC training set to 15,016 participants and found that recruited participants were right in 66 percent of attempts, against 65 percent for the leading DFDC model on the full 4,000-video holdout set (PNAS 2022). The clip-level features then explain which videos were hard. On the 14 videos that were grainy, blurry or dark, the recruited crowd was right on 8; on the other 36 it was right on 29. A flickering face, the one temporal artefact in their list, raised recruited participants' accuracy by 24.2 percentage points. The two videos that beat both the crowd and the model were "extremely blurry" and "filmed from a low angle" with glare on the actress's glasses. Low quality is not a sign of a fake; it removes the signs.

The same study tested what happens when ordinary face perception is disrupted. Clips shown upside down cut recruited participants' accuracy by 4.3 percentage points and the model's by 12.1; splitting the face horizontally cut recruited accuracy by 6.1 points. The authors conclude that "manipulations designed to disrupt visual processing of faces hinder human participants' performance", which means much of what people detect is holistic, a face that reads as wrong, rather than a nameable tell. Köbis et al. (iScience 2021) add the direction of the error: on 16 clips drawn from the 3,000 hardest DFDC videos, 210 participants scored 57.6 percent and guessed "authentic" 67.4 percent of the time, so the default error is to accept a fake.

05Do the signs survive in synthetic still faces?

For GAN faces, the measured answer is no. Nightingale and Farid (PNAS 2022) showed 315 participants 128 faces each from 400 StyleGAN2 faces and 400 matched real ones; accuracy was 48.2 percent, below the 50 percent chance line. A second group of 219 participants given training and trial-by-trial feedback reached 59.0 percent, with "no improvement in accuracy over time" during the session, and a third group rated the synthetic faces more trustworthy, 4.82 against 4.48 on a seven-point scale. Training and feedback added about eleven points and then stopped helping.

The generator papers explain why the tells expire. The StyleGAN2 authors "expose and analyze several of its characteristic artifacts", including the water-droplet artefact caused by instance normalisation, "and propose changes in both model architecture and training methods to address them" (arXiv 1912.04958). A sign listed in a guide written against one generator is a defect its successor was built to remove. The which face is real game and its studies page records how that played out for the best-known training site.

06Do detectors see the same signs people see?

Partly, and the differences are informative. FaceForensics++ reports that "our automated detectors outperform human performance by a large margin": XceptionNet trained on all four methods reached an accuracy of 99.26 percent on raw frames and 81.00 percent at c40 against the study's 68.69 and 58.73. Detectors also caught the reenactments people missed, since the per-method accuracies in the paper's Table 5 stay above 90 percent at c23 for all four methods.

Groh et al. found the model and the crowd "similarly accurate, while making different kinds of mistakes", and that inverted video cost the model 12.1 points against 4.3 for recruited participants. The practical reading is that a human eye and a detector score are two weak and partly different witnesses, which is the order the checking a face video page gives them. The detection techniques compared page lists what each detector family looks for, and deepfake technology describes the generators that produce the artefacts. No single sign, and no single benchmark score, predicts whether a given clip is a fake.