Can people spot AI faces? What the "which face is real" studies found

This page is part of the deepfake detection hub. It covers fully generated faces, the kind the AI-generated faces page describes; the evidence for face swaps in video is different and is covered below.

01The "which face is real" game and what it tests

Which Face Is Real (whichfaceisreal.com) was built in 2019 by Jevin West and Carl Bergstrom at the University of Washington as part of their Calling Bullshit project. Each round shows two faces, one a real photograph and one produced by a generative network, and asks the player to pick the real one. The site's stated aim is "to make you aware of the ease with which digital identities can be faked, and to help you spot these fakes at a single glance", and its companion page teaches the tells of the generator of the day: mismatched earrings, melted backgrounds, hair that dissolves into skin, asymmetric glasses.

The game is a teaching tool, not a measurement. It uses a two-alternative forced choice, which is easier than judging one face alone; it does not publish accuracy figures; and its tells belong to the 2019 generator. What it does establish is the shape of the task that the studies then measured.

02The peer-reviewed studies

Nightingale and Farid selected 400 faces synthesised by StyleGAN2, balanced across gender, age and four racial groups, and for each one found a matching real face from the 70,000-image set the generator was trained on, matched for gender, age, race and overall appearance. Faces with obvious rendering artefacts were removed before the experiments, which the authors note makes the task harder but realistic, since a deceptive use would not ship a face with visible errors.

In experiment 1, 315 participants each classified 128 of those 800 faces as real or synthetic, one at a time, with no feedback. Average accuracy was 48.2% (95% confidence interval 47.1% to 49.2%), with no bias toward either answer. White faces were the least accurately classified, which the authors attribute to their over-representation in the training data and hence greater realism.

In experiment 2, 219 new participants saw a tutorial on the artefacts that give synthetic faces away and received feedback after every trial. Accuracy rose to 59.0% (57.7% to 60.4%), but there was no improvement across the session: 59.3% on the first 64 faces, 58.8% on the second 64. The authors' reading is that feedback has a limited effect because some synthetic faces contain no perceptible artefact to learn.

In experiment 3, 223 participants rated trustworthiness on a seven-point scale. Synthetic faces averaged 4.82 and real faces 4.48, a 7.7% difference the authors report as significant, with the three most trusted faces all synthetic and the four least trusted all real.

The FaceForensics++ paper measured the video case for manipulated, not generated, faces. Its user study with 204 participants asked them to judge manipulated and original video frames at the dataset's three compression levels: 68.69% correct on raw video, 66.57% on high quality and 58.73% on low quality. On the same material the best automated detector scored 99.26%, 95.73% and 81.00%. People do better on 2019 face swaps than on 2022 generated stills, and compression hurts both people and detectors.

03Humans versus detectors

The comparison only holds inside one study, on one set of fakes. Within FaceForensics++, detectors beat people by 30 to 22 points depending on compression. On circulated fakes the detectors themselves fall: the in the wild page gives the 2024 figures, where the best off-the-shelf open-source video model scored 0.63 AUC. No published study has run people and detectors on the same in-the-wild set under matched conditions, so a general statement that machines beat humans, or the reverse, is not supported. The results page lists the detector side with conditions.

04Fully generated faces versus face swaps

The two kinds of fake leave different evidence. A generated still has no original to compare against and, after the obvious artefacts are culled, no seam; the Nightingale and Farid result shows there is often nothing left for the eye to find. A face swap in a video has a boundary where the new face meets the old head, a frame-to-frame consistency it must keep, and a source somewhere; the FaceForensics++ human scores are higher partly because of that. The kinds of face deepfake page separates swap, reenactment, attribute editing and full synthesis, and says which datasets cover each.

05What helps a person

Training helped in the study, by about eleven points, and then stopped helping. Two-alternative comparison, as in the game, is easier than judging one image alone. Neither gets a person far above chance on good synthetic stills. What does work is not perceptual: finding where the image was first posted, whether it carries content credentials, and whether anyone else has an independent copy. The checking a face video page puts those steps in order of evidence and places looking closely where the studies say it belongs, near the bottom.