AI or human? Why synthetic faces are rated more trustworthy than real ones

This page belongs to the deepfake detection hub and follows the studies of people spotting synthetic faces, which cover the same paper's first two experiments on accuracy. This one is about the third experiment, on trust, and about the explanation the authors offer for it.

01What did the trust experiment measure?

The paper, "AI-synthesized faces are indistinguishable from real faces and more trustworthy" (doi 10.1073/pnas.2120481119), set three experiments on one pool of 800 faces, half real and half synthetic. In the third, "223 participants rated the trustworthiness of 128 faces taken from the same set of 800 faces on a scale of 1 (very untrustworthy) to 7 (very trustworthy)". No participant was told which faces were synthetic, and the rating task did not ask.

Averaging the ordinal ratings, the real faces scored 4.48 and the synthetic faces 4.82. The authors write that "although only 7.7% more trustworthy, this difference is significant", reporting t(222) = 14.6, P < 0.001 and an effect size d of 0.49. The gap is small on the scale and consistent across raters, which is why a brief report in a general science journal led with it.

02How were the faces chosen?

The synthetic half was 400 faces produced by StyleGAN2, selected for balance: 200 women and 200 men, and 100 each described as Black, Caucasian, East Asian and South Asian, across a range of apparent ages. Faces with obvious rendering artefacts were removed, which the authors note makes the task harder and matches how a deceptive synthetic face would actually be chosen.

Each synthetic face was then paired with a real one. A convolutional network descriptor turned each face into a 4,096-dimensional vector, and that vector was "compared with all other facial representations in the data set of 70,000 real faces to find the most similar face" in the database the generator had been trained on, subject to the same selection rules: plain background, unobstructed face, in focus, no writing on clothing. The matching matters for the trust result. A synthetic face and its real partner share gender, race, age and general appearance, so the gap between 4.48 and 4.82 is not a gap between two differently composed sets of people.

03Did gender, race or smiling explain the gap?

Each was checked. Women's faces were rated "significantly more trustworthy than men, 4.94 as compared to 4.36", with d = 0.82, a larger effect than real against synthetic; because the two halves were matched on gender, this difference runs through both halves equally and cannot produce the gap. Across race, "Black faces were rated more trustworthy than South Asian faces, but, otherwise, there was no effect across race", and the authors call that effect small. Expression was checked by counting: 65.5 percent of the real faces and 58.8 percent of the synthetic faces were smiling, so if smiling raised trust it should have favoured the real half, and the authors conclude that "facial expression alone cannot explain why synthetic faces are rated as more trustworthy".

The extremes point the same way. In the paper's Figure 3, the three most trustworthy faces in the whole set are synthetic and the four least trustworthy are real.

04Why would an AI face look more trustworthy?

The authors' explanation is typicality: "this may be because synthesized faces tend to look more like average faces which themselves are deemed more trustworthy". The reference they give is Sofer, Dotsch, Wigboldus and Todorov (Psychological Science, 2015), who built a continuum of faces running from typical to highly attractive and found that "trustworthiness judgments peak around the typical face", while "perceived attractiveness increases monotonically past the typical face" (doi 10.1177/0956797614554955). Trust and attractiveness usually move together, which had hidden the typicality effect in earlier work; separated, trust favours the average and attractiveness does not.

On that account, a generator trained to produce plausible faces from a large set of real ones will tend to produce faces near the centre of that set, and faces near the centre are the ones people trust. The PNAS paper presents this as a likely cause rather than a tested one; it did not measure typicality directly, and the claim about where generator outputs sit is the authors' inference. What the data establish is the gap itself and the ruling out of gender, race and expression as its source.

05Can trust be used to tell AI from human?

The experiment was designed to find out. Faces trigger trait inferences within milliseconds, and the authors reasoned that "if not, then a perception of trustworthiness could help distinguish real from synthetic faces": if synthetic faces failed to trigger the usual judgement, that failure would be a tell. The result went the other way. Synthetic faces triggered the judgement, and more strongly, so trust is not a cue; it is a liability.

The accuracy experiments in the same paper fix the baseline. In experiment 1, 315 participants classified faces as real or synthetic with 48.2 percent accuracy, below chance; in experiment 2, 219 participants given training and feedback reached 59.0 percent and did not improve over the session. A face that is slightly more trusted and cannot be identified as synthetic is the combination the paper's title describes. The which face is real game page covers the public version of the task that preceded the study.

06What this means for checking a face

A synthetic still face carries no reliable visual tell for an untrained viewer and, on this evidence, a small advantage in first impressions. That is the mechanism behind the fake-profile use the paper's discussion names, and it is why the checking a face in order of evidence page asks where an image came from before asking how it looks. For video, where the alteration has to hold frame by frame (the how a deepfake video is scored page explains the scoring), the human numbers are different and the studies are separate; the human or AI studies on video page collects them, and the detection results on circulating fakes page gives the detector side, where open-source models' AUC fell by about 50 percent on 2024 material.

07What the study does and does not show

The result is one experiment, on StyleGAN2 still faces generated before 2022, with 223 raters and a 7-point self-report scale. It does not cover diffusion-model faces, video, or faces chosen without the artefact culling; it does not test whether the trust difference changes behaviour; and the typicality explanation is cited rather than measured. The number to quote is the pair 4.48 and 4.82, with the scale, the participant count and the date attached. No single study score predicts how people respond to the next generator's faces, any more than a single benchmark score predicts how a detector performs on unseen manipulations.