Which Face Is Real? What the game tests, teaches, and what studies found

This page is part of the deepfake detection hub and centres on the game itself: what a two-alternative choice measures, what its learn page teaches, and why the tells it lists do not last. The peer-reviewed studies of people spotting AI faces are covered on their own page and only summarised here.

01What is the Which Face Is Real game?

The site's methods page states the design in one sentence: "we present pairs of images: a real one from the FFHQ collection, and a synthetic one, as generated by the StyleGAN system", with the synthetic images taken from thispersondoesnotexist.com (whichfaceisreal.com, methods). The generator is not the authors' work, and the page says so: NVIDIA released the StyleGAN code and trained weights in February 2019, trained on CelebA-HQ and FFHQ, the latter "70,000 facial photographs posted to Flickr under a creative commons license".

The about page gives the purpose: "Our aim is to make you aware of the ease with which digital identities can be faked, and to help you spot these fakes at a single glance" (whichfaceisreal.com, about). It is a teaching tool with a 2019 copyright line, not a benchmark, and it keeps no published leaderboard. The faces behind it come from the StyleGAN paper (Karras, Laine and Aila, CVPR 2019), which introduced the FFHQ dataset in the same work (arXiv 1812.04948).

02What does a two-alternative forced choice test?

Showing two images and asking which is real is a two-alternative forced choice. Exactly one of the pair is synthetic, so a player who sees any difference at all can use it, and a guess scores 50 percent. The task measures relative judgement: is this face more plausible than that one?

The controlled studies used a harder design. In Nightingale and Farid's first experiment, "315 participants classified, one at a time, 128 of the 800 faces as real or synthesized" (PNAS 2022, full text). With no reference image next to it, each face has to be judged against the viewer's memory of what real faces look like, and chance is still 50 percent. The two task types are not interchangeable, so a high personal score on the game and the 48.2 percent from the study do not contradict each other; they measure different things on different faces, and only the study controls who played, how many times and with what feedback.

03What does the learn page teach?

The site's learn page lists the tells that StyleGAN left in 2019, each with example crops. The list below is its headings, with its own description of each.

The page also lists signs that a photograph is real: "symmetric eyeglasses and earrings", "actual human-looking companions" at the edge of the frame, and "detailed backgrounds especially if there is readable text" (whichfaceisreal.com, learn). It credits the tells to a 2018 tutorial by Kyle McDonald. Every item on the list is a failure of the generator, not a property of synthetic faces in general, which is the reason the list dates.

04Why do the tells age?

The authors said so themselves on the water-splotch entry: "We can't expect future algorithms to have this problem". The next version of the generator removed it. The StyleGAN2 paper, submitted in December 2019, opens by stating that the authors "expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them" (arXiv 1912.04958). A tell published in February was an engineering target by December.

The learn page records its own first casualty. At launch the authors offered what they called a silver bullet: "The StyleGAN algorithm is unable to generate multiple images of the same fake person", so a second photo of the same person was proof. Their update reads: "Well, it took three months, not a year or two", citing a 2019 paper that animated a face from a single image. The about page had already set the expectation: "it may be only a few years until humans fall behind in the arms race between forgery and detection". The studies that followed measured when that happened for still faces.

05What did the controlled studies find?

Nightingale and Farid used 400 StyleGAN2 faces, 200 women and 200 men, each matched to a real face. Untrained participants scored 48.2 percent (95 percent confidence interval 47.1 to 49.2), which the paper calls "close to chance performance of 50%". A second group of 219 participants given training and trial-by-trial feedback reached 59.0 percent, and "there was no improvement in accuracy over time" within the session. A third group of 223 rated the synthetic faces more trustworthy, 4.82 against 4.48 on a seven-point scale, "only 7.7% more trustworthy" but a reliable difference. The paper's explanation for the hardest category is a training-data effect: White faces "are overrepresented in the StyleGAN2 training dataset and are therefore more realistic".

For manipulated video rather than generated stills, the FaceForensics++ user study of 204 participants found accuracy of 68.69 percent on raw video, 66.57 percent on high quality and 58.73 percent on low quality (arXiv 1901.08971). Video swaps leave more to see than a StyleGAN portrait does, and compression hides it. The human or AI page sets these numbers beside the other studies on the same question.

06What the game can and cannot tell you

The game shows that a motivated person can learn the 2019 generator's failures quickly, which is what it was built for. It cannot say how often people are fooled, because its players choose to play, repeat rounds, and see the answer after each one; none of that holds for a face met in a profile or a message. The controlled figure for that situation is the 48.2 percent above, and the reason detection is left to measured models rather than to the eye is set out in how deepfake detection works.

Two related tasks are easy to confuse with this one. Telling a live person from a replayed face at a camera is presentation attack detection, with its own standard and error rates, on the PAD standard explained page. And the model-side version of the ageing tells, a detector trained on one generator meeting the next, is quantified on why detectors fail in the wild.

The quotable caveat: a list of visual tells describes one generator at one date, and the 59.0 percent reached with training in the PNAS study is the measured ceiling for the eye on StyleGAN2 faces.