Human deepfake detection study results: every figure in one table

This page is part of the deepfake detection hub. It is the reference table behind the can people spot AI faces page, which discusses the still-image studies, and the human or AI page, which compares people with detectors on video.

01Why a chance level sits beside every figure

A detection accuracy means nothing without the rate a coin would score. Every study here used stimuli split half real and half fake, or a two-alternative choice between one real and one fake clip, so chance is 50 percent throughout, and a result is only evidence of skill by the amount it clears that line. Nightingale and Farid's first experiment, at 48.2 percent with a 95 percent confidence interval of 47.1 to 49.2, is the clearest case: the interval excludes 50, so people were slightly worse than guessing, not merely near it.

Task design changes what accuracy measures. A single-stimulus task asks whether one face or clip is fake; a paired task shows one real and one fake and asks which is which, which is easier because the pair shares everything but the manipulation. Groh et al. ran both designs on the same source material and the numbers differ, so the table keeps the design in its own column.

02The table

Sources: arXiv 1901.08971 section 4 and Figure 4; arXiv 2106.07226 sections 3 and 4; iScience 24(11) 103364; PNAS 119(1) e2110013119; PNAS 119(8) e2120481119. All read 9 November 2026.

03How do the still-image studies compare?

Three studies used generated still faces and they agree on direction. Lago et al. found accuracy falling with each generator generation, from 80 percent on PGGAN faces to 26 percent on StyleGAN2 faces, while real faces were correctly called real only 56 percent of the time. Nightingale and Farid, using StyleGAN2 alone and a balanced design, found 48.2 percent, below chance. Training helped a little and then stopped helping: the second experiment's 59.0 percent came with feedback after every trial and "no improvement in accuracy over time". The FaceForensics++ study is the outlier at 68.69 percent, and its stimuli explain why: they were frames from 2019 manipulation methods at raw compression, and the figure fell to 58.73 percent on the c40 frames of the same videos.

The confidence finding is shared. Köbis et al. report that people "overestimate their own detection abilities", and Lago et al. found a negative correlation between accuracy and confidence on StyleGAN2 images: "the more a face is inaccurately perceived as real, the higher the observer's confidence".

04What did the video studies find?

The two video studies drew from the same source, the Deepfake Detection Challenge, and landed between 57 and 69 percent. Köbis et al. chose the 3,000 DFDC videos hardest for classifiers and found 57.6 percent, with a strong bias toward calling videos authentic, 67.4 percent of answers against a true rate of 50 percent. Groh et al. sampled 50 videos from the DFDC holdout set and found 66 percent for recruited and 69 percent for nonrecruited participants on single videos; their comparison model scored 80 percent on those 50 videos and 65 percent on the full 4,000 video holdout, so people and model were, in the authors' words, "similarly accurate, while making different kinds of mistakes".

Aggregating people changed the picture more than training did. The crowd mean of nonrecruited participants reached 80 percent, equal to the model, and 86 percent for those who had seen at least 10 videos. Showing people the model's prediction raised recruited participants from 66 to 73 percent on average, but a confident wrong prediction pulled them down with it, from 56 to 34 percent on one video and 70 to 55 percent on another. The human or AI page takes up that comparison in detail.

05What the studies do not measure

None of these studies tested fakes collected from the internet, and none tested audio. The still-image studies end at StyleGAN2, released in 2019, and the video studies use DFDC, made with paid actors who agreed to have their likenesses modified; neither covers diffusion models or the face swap applications of 2024. The deepfake detection protocols page makes the same point for machine detectors: a score is bound to the stimuli it was measured on, and the Deepfake-Eval-2024 benchmark reports that even fine-tuned detectors "do not yet reach the accuracy of deepfake forensic analysts" without putting a number on the analysts.

Participants also differ from the people who meet deepfakes in practice. Four of the five studies used online participants who knew they were being asked to judge faces or clips; the FaceForensics++ participants were mostly computer science students. A viewer scrolling a feed has no warning that a clip might be fake, and no study here measures that case. The error vocabulary differs from machine testing too: a presentation attack test reports APCER and BPCER per attack type, while these studies report one accuracy and, at most, a bias direction.

06How to read a human study before quoting it

Four checks cover most misreadings. First, the chance level: 50 percent for every study above, but a study with unequal real and fake counts would shift it. Second, the task: a paired choice is easier than a single judgement, and Groh et al.'s two experiments should not be compared as one number. Third, the stimuli: generator, year, compression and whether frames or full videos were shown, since the FaceForensics++ figure dropped 10 points from raw to c40 on the same material. Fourth, who the participants were and whether they were trained, because Nightingale and Farid's trained group scored 10.8 points above their untrained one on identical faces.

The same discipline applies to the 1:1 versus 1:N recognition figures on the face recognition side: an error rate travels with its threshold and its gallery, as an accuracy travels with its chance level and its stimuli. A number quoted without those conditions is a number from a different study.