Face recognition bias: what was measured, where, and what causes it

This page is part of the face recognition section and is written for a reader who has met the phrase and wants to know what stands behind it. The demographic differentials page works through the NIST figures in detail; this page separates the two studies, names the causes the evidence supports, and lists what a bias claim has to specify.

01What does "face recognition bias" mean?

In the evaluations that produced the figures, bias is not a property of a system in the abstract but a measured gap. NIST's report defines differential performance as a "difference in the genuine or imposter [score] distributions" and adds that such differences "are inconsequential unless they prompt a differential outcome", an outcome being what happens when a score is compared with the operator's threshold (NISTIR 8280). A genuine score below the threshold is a false negative; an impostor score at or above it is a false positive. A demographic differential is a difference in one of those two rates between groups at the same threshold.

That definition carries four parts, and each can change the answer: which algorithm, which error, which threshold, and which groups are compared. A system can have a large false positive differential and no false negative differential; it can have both at one threshold and neither at another; and two algorithms from different developers can disagree in sign. "Face recognition is biased" compresses all four parts into one word, and the studies behind the word do not support the compression.

02Which two studies are usually meant?

Two studies supply most of the public figures, and they measured different systems. Gender Shades (Buolamwini and Gebru, 2018) tested three commercial services that guess the gender of a face from a photograph; it did not test whether a system could match two photographs of one person (PMLR 81). NISTIR 8280 (Grother, Ngan and Hanaoka, 2019) tested 189 recognition algorithms on verification and identification, the tasks that compare faces, using 18.27 million images of 8.49 million people from four United States government image collections.

NIST drew the line itself. The report notes that "much of the discussion of face recognition bias in recent years cites two studies showing poor accuracy of face gender classification algorithms on black women", that "those studies did not evaluate face recognition algorithms", and it recommends that "reporting of bias should include information about the class of algorithm evaluated". The two findings are both real; they are findings about different software.

03What did Gender Shades measure?

Gender Shades built the Pilot Parliaments Benchmark, 1,270 photographs of members of parliament from Rwanda, Senegal, South Africa, Iceland, Finland and Sweden, labelled by a dermatologist on the Fitzpatrick skin type scale and by binary gender, then queried three commercial classifiers in April and May 2017. Every classifier erred most on darker-skinned women, with error rates from 20.8 to 34.7 percent, and least on lighter-skinned men, where the highest error was 0.8 percent; the largest gap between the best and worst group was 34.4 points. The Gender Shades study page gives every cell of the results table.

The study's second finding is about data rather than software. Two widely used face benchmarks, IJB-A and Adience, were 79.6 percent and 86.2 percent lighter-skinned subjects by the authors' labelling. That observation applies to any face system trained or tested on similar collections, and it is the part of Gender Shades that carries into recognition.

04What did NIST find in face recognition?

The headline is the asymmetry between the two errors. In NISTIR 8280, false positive rates across demographics "often vary by factors of 10 to beyond 100 times", while false negative rates "vary often by factors below 3" and are more algorithm specific. With application photographs, false positives were highest for West and East African and East Asian faces and lowest for Eastern European faces; for a number of algorithms developed in China the East Asian effect reversed. False positives were higher for women than men across algorithms and datasets, an effect the report calls smaller than the race effect, and higher for the elderly and for children. In domestic mugshots, the highest false positives were in American Indians.

The findings are not universal. The differentials "exist broadly, across many, but not all, algorithms tested", and the report states that "some developers supplied highly accurate identification algorithms for which false positive differentials are undetectable". On false negatives it goes further: "universal statements pertaining to algorithms false negative rates across sex and age are not supported". The NISTIR 8280 report page follows the document section by section.

05What causes the differentials?

Less is established than the public discussion assumes, and NIST says so. The report lists among the things it did not do: "we did not train algorithms", and "we did not make efforts to explain the technical reasons for the observed results, nor to build an inferential model of them". Three causes are nonetheless supported by its own observations. First, training data: the report describes "the location of the developer as a proxy for the race demographics of the data they used in training" and notes that algorithms from China gave low false positive rates on East Asian faces, which points at the composition of the training set rather than at the task. Gender Shades found the same imbalance in public benchmarks.

Second, image quality. NIST found that with high quality application photographs false negative differentials were hard to measure at all, and concluded that "better image quality reduces false negative rates and differentials"; with lower quality border crossing images false negatives rose, and rose more for people born in Africa and the Caribbean. Third, the fixed threshold. Because "the vast majority of biometric systems are configured with a fixed threshold against which all comparisons are made", a threshold set to give one false match rate on the whole population gives a different rate on each group. The differential exists at the operating point; it is not visible in a curve.

06Why does a single bias number mislead?

No single bias figure describes face recognition; a differential belongs to one algorithm, one error type, one threshold and one pair of groups. The two errors also harm different people. A false negative means a person is not recognised as themselves: a delay, a second check, a denied entry. A false positive means a person is matched to someone else: in a one to many search, a wrongful place on a candidate list. NIST reports the two separately "because the consequences of each type of error are of importance to different communities", and the false positive rate page explains why the false positive side carries the larger differentials.

The threshold matters as much as the error type. NIST criticises studies that report false negative rates at fixed false positive rates, because that method hides "excursions in false positive rates" between groups. A system quoted as "99 percent accurate" or as having "a false match rate of one in a million" is quoting a population average; the per-group rate at that same threshold is the bias measurement, and it is, in NIST's words, "rarely done".

07Is deepfake detection measured the same way?

Not yet at the same scale. Deepfake-Eval-2024, the 2024 in-the-wild detection benchmark, labelled 10 percent of its visual data by perceived race and found it 54.6 percent white and 45.4 percent non-white, but reports no per-group error rates for the detectors it tested (arXiv 2503.02857). The Deepfake-Eval-2024 benchmark page describes what it does report. A detector is a classifier, so the Gender Shades method of per-group error at a fixed threshold applies to it directly; the NIST method of per-group false match rates applies only where a system compares two faces.

08What must a bias claim specify?

A statement about face recognition bias is checkable when it names five things. The class of algorithm: gender classification, verification or identification, because NIST and Gender Shades measured different ones. The error: false positive or false negative, because they differ by an order of magnitude in how much they vary. The threshold, or the overall rate that set it. The groups compared, with the image set they came from. And the algorithm, since the differentials "exist broadly, across many, but not all". A claim with all five, such as "at a threshold giving an overall false match rate of 0.0003 on application photographs, algorithm X's false match rate was 100 times higher for one region of birth than another", is a measurement. A claim with none of them is a slogan, whichever side it argues.