False positive rate: definition, formula and why it decides deployment

This page belongs to the deepfake detection hub and is the general definition behind the scoring pages. The how detectors are scored page places the false positive rate beside AUC, EER and accuracy; this page defines it on its own and follows it into face recognition.

01What is the false positive rate?

A binary classifier makes two kinds of mistake. Fawcett's introduction to ROC analysis (Pattern Recognition Letters, 2006) lays out the four outcomes: a positive item classified positive is a true positive, a positive classified negative is a false negative, a negative classified negative is a true negative, and "if it is classified as positive, it is counted as a false positive". Counting the four over a test set gives a two by two confusion matrix, which Fawcett also calls a contingency table.

The false positive rate, which Fawcett notes is "also called false alarm rate", is the bottom left cell divided by the bottom row: false positives over all genuine items. In words, out of every hundred genuine items, how many did the system flag. The rate runs from 0, no genuine item flagged, to 1, every genuine item flagged, and it is tied to one threshold. Move the threshold and the same system has a different false positive rate.

02How does it differ from accuracy and precision?

The three metrics divide by different rows and columns of the same matrix. Accuracy divides the correct cells by everything, so it depends on how many fakes the test set contains; Fawcett opens his tutorial with "the realization that simple classification accuracy is often a poor metric for measuring performance". Precision divides true positives by everything flagged, so it mixes the two classes. The false positive rate divides by genuine items only, which is why Fawcett can state that ROC curves, drawn from the false positive rate and the true positive rate, "are insensitive to changes in class distribution": a test set with ten times more fakes gives the same false positive rate for the same threshold.

Its complement has a name too. Specificity is true negatives over all genuine items, and Fawcett writes it as 1 minus the false positive rate. A paper that reports specificity of 0.98 is reporting a false positive rate of 0.02 at that threshold, and the two words should not be read as different measurements.

03Why does the operating point matter more than AUC?

AUC summarises every threshold at once, and that is its weakness for a deployment that uses one. The AUC score explained page covers the probability reading; the relevant point here is Fawcett's warning that "it is possible for a high-AUC classifier to perform worse in a specific region of ROC space than a low-AUC classifier". A system in use sits at one point on its curve, and the false positive rate at that point is what its operators experience as false alarms.

Deepfake detection papers mostly report AUC. DeepfakeBench scores its 15 detectors by frame-level AUC, average precision, equal error rate and accuracy, and the deepfake detection models ranked page orders them by that AUC; no false positive rate at a named threshold appears in its main table. Deepfake-Eval-2024 reports AUC, F1 and accuracy in the paper and keeps "precision, recall, false positive rate, and false negative rate" for the supplement (arXiv 2503.02857). A reader who needs the false alarm rate of a detector therefore usually has to find a threshold the authors chose, or compute it from released scores.

04What is the false positive rate called in face recognition?

Face recognition splits the same quantity by task and uses the vocabulary of ISO/IEC 19795-1:2021, Biometric performance testing and reporting, Part 1. In one to one verification the genuine items are impostor comparisons, pairs of different people, and the error is a false match. The NIST FRTE 1:1 page defines it directly: "FMR is the proportion of impostor comparisons at or above that threshold" (NIST FRTE 1:1). In one to many identification the genuine items are non-mated searches, probes of people not in the gallery, and the NIST 1:N page defines the false positive identification rate as "the proportion of non-mated searches producing one or more candidates above threshold" (NIST FRTE 1:N). The FMR, FNMR, FPIR and FNIR page sets out all four terms with their mates.

The operating points NIST uses show how small these rates are in practice. The 1:1 leaderboard orders algorithms by false non-match rate at a false match rate of one in a million; its demographic summaries fix the threshold to give an overall false match rate of 0.0003 or 0.00001; and the 1:N leaderboard fixes the false positive identification rate at 0.003. A deepfake detector evaluated at one percent false positives would be between roughly 3 and 10,000 times looser than those points.

05Why does NIST fix the threshold rather than the rate?

Because that is how systems are run. NISTIR 8280, the demographic effects report, states that "the vast majority of biometric systems are configured with a fixed threshold against which all comparisons are made" and criticises academic work that reports "false negative rates at fixed false positive rates rather than at fixed thresholds, thereby hiding excursions in false positive rates" (NISTIR 8280). At a fixed threshold the false positive rate is free to vary between groups, and the report found that across demographics false positive rates "often vary by factors of 10 to beyond 100 times" while false negatives vary "often by factors below 3".

That asymmetry is the reason the false positive rate carries the demographic argument. A system tuned to one false match rate on the whole population can have a far higher rate on one group, and only a per-group false positive rate at the deployed threshold reveals it. The face recognition bias page explains what a bias claim has to specify, and the per-group false positive rate at a fixed threshold is the first item.

06Why does the false positive rate decide deployment?

A false positive rate is meaningless without its threshold and its denominator, and once both are fixed it becomes a count. A rate of 0.001 means one genuine item in a thousand is flagged; applied to a million genuine comparisons a day, it means a thousand false alarms a day, each of which someone must handle. NISTIR 8280 puts the two errors in those terms: in one to one access control "false negatives inconvenience legitimate users; false positives undermine a system owners security goals", and in a one to many search a false positive is a person wrongly placed on a candidate list.

The same arithmetic governs a deepfake detector that acts on its verdict. Each false positive is a real recording treated as a forgery, so a platform that screens millions of uploads cannot run at the false positive rate a benchmark paper tolerated, and a detector whose AUC was earned mostly in the high false positive region of its curve will look far weaker once the threshold is set where a deployment needs it. That is why the practical reading of any detector result is the true positive rate at a stated, low false positive rate, not the AUC.

07How should a false positive rate be read in a paper?

Four conditions have to be present before the number means anything. The threshold, or the target rate that set it. The denominator, meaning what counted as genuine: real clips, impostor pairs or non-mated searches. The dataset the genuine items came from, because real footage from one source is not real footage from another. And whether the rate is pooled or reported per group. A sentence such as "a false positive rate of 0.0003 on visa photographs at a threshold set on the whole population, with per-country rates in the annex" carries all four. A sentence such as "a 2 percent false positive rate" carries none, and cannot be compared with any other.