Face recognition bias: what NIST measured about demographic differentials

This page is part of the face recognition section and takes its figures from one source, NISTIR 8280, because it is the only evaluation of that scale run on operational images by a body with nothing to sell. The NIST evaluations page explains how such reports are produced.

01What "bias" means in a face recognition table

A verification system makes two kinds of error, and the face verification metrics page explains why they are reported at a fixed threshold. A demographic differential is the finding that, at one threshold, one of those error rates is higher for one group than for another. The two errors have different consequences. A higher false non-match rate means members of a group are more often rejected when they are who they say they are: inconvenience, delay, a second check. A higher false match rate means members of a group are more often matched to someone else: in a one-to-many search, a higher chance of being wrongly placed on a candidate list. Any statement about bias has to say which error, at which threshold, on which images, for which algorithm, or it is not a measurement.

02What the NIST demographic effects report measured

NISTIR 8280, "Face Recognition Vendor Test Part 3: Demographic Effects", published on 19 December 2019 by Grother, Ngan and Hanaoka, processed 18.27 million images of 8.49 million people through 189 mostly commercial algorithms from 99 developers (NISTIR 8280). The images came from four operational sources: domestic mugshots, immigration application photographs, visa photographs and border crossing photographs, each with demographic labels from the records they were drawn from. The report measured, for each algorithm and each dataset, how the false match rate and false non-match rate varied by sex, age and race or country of birth at fixed thresholds, using the 1:1 terms FMR and FNMR and the 1:N terms FPIR and FNIR.

The evaluation's design matters as much as its findings. It tested the algorithms as submitted, on images the developers had never seen, so the differentials are properties of the algorithms, not of a test chosen to produce them. And it tested many algorithms, which is why its conclusions are about the range of behaviour rather than about one system.

03False match versus false non-match differentials

The central finding is the asymmetry. False positive differentials were much larger than false negative differentials, and they were widespread: false match rates "often vary by factors of 10 to beyond 100 times" across demographics, while false non-match rates varied "often by factors below 3". With application photographs, false positives were highest in West and East African and East Asian people and lowest in Eastern European individuals, with the effect reversed for a number of algorithms developed in China. False positives were higher in women than in men, an effect the report describes as smaller than the race effect, and were elevated in the elderly and in children. The report also notes that some highly accurate identification algorithms showed undetectable false positive differentials, which is the finding most often left out of summaries: the differentials are common, not universal, and the best algorithms did not show them.

Read as this site reads every number, each of those statements carries its dataset (application photographs for the race finding), its error type (false positives) and its qualifier (many but not all algorithms). The report runs to many pages of per-algorithm tables precisely so that no single sentence has to stand for all of them.

04Why training data matters

The report measures the effect and does not attribute it, but the reversal it observed, with several algorithms developed in China showing the lowest false positive rates on East Asian faces, points where most researchers point: at the composition of the training data. Face recognition models are trained on millions of labelled faces, and the large public training sets were assembled by downloading photographs of named people from the web, with whatever demographic balance that produced. The datasets and consent page describes how those sets were built and what happened to several of them; the WebFace260M page describes the largest, whose paper reports its scale but, like its predecessors, was gathered from the internet rather than sampled to a demographic design.

05What the numbers do not say

The report does not say that face recognition is biased in general, that any particular deployment is, or that every algorithm shows the effect; it says which algorithms showed which differentials on which images at which thresholds, in 2019. It does not say what causes the differentials, though it notes where they were absent. It does not cover photographs taken outside controlled settings, so it does not speak to performance on social media images. And a differential in false match rate is not the same as a difference in accuracy: an algorithm can have a low false match rate for every group and still show a differential of a factor of ten between two of them. Each of those limits is a field that must travel with the finding, exactly as test set, protocol and date travel with a score on this site's results pages.

06Does deepfake detection have the same problem?

Detection is scored on real and manipulated faces rather than on pairs of people, so the question becomes whether a detector's false positive rate on genuine video differs by the demographics of the person shown. No evaluation of the scale or independence of NISTIR 8280 has been published for deepfake detectors. The nearest evidence is indirect: the human study on the can people spot AI faces page found that White synthetic faces were the hardest for people to classify, which its authors attribute to over-representation in the generator's training data, and the in-the-wild results on the in the wild page show detectors failing on material unlike their training sets. Both suggest that detector error rates depend on who and what the training data contained. Measuring it, per group, at a fixed threshold, on operational material, has not been done in public.