LFW dataset: 13,233 images, 5,749 people, 6,000 pairs, why it saturated

This page is part of the face recognition section. It describes the benchmark that defined face verification for a decade; the deep learning era of face recognition page follows what the networks that saturated it changed.

01What is the LFW dataset?

LFW is a set of face photographs labelled with the name of the person shown, released in 2007 by Gary Huang, Manu Ramesh, Tamara Berg and Erik Learned-Miller under the title "Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments". The word "unconstrained" was the point: rather than the controlled pose and lighting of a studio capture, LFW's images carry whatever pose, lighting, expression and occlusion the original photographs had, which is what a deployed system meets.

The counts on this page come from the InsightFace datasets README, which lists LFW as 5,749 identities, 13,233 images and 6K pairs and cites the 2007 report (InsightFace README). The dataset's own project page could not be reached when this page was checked, so figures the README does not carry, such as how many identities have more than one image, are not stated here.

02How does the 6,000-pair protocol work?

LFW is a verification benchmark, not an identification one. A system is shown two images and asked whether they show the same person, and its accuracy is the share of 6,000 such pairs it gets right, some pairs showing one person and some showing two. The output is a single percentage, which made LFW easy to report and easy to compare across papers for a decade. The face verification metrics page explains why later benchmarks replaced that single percentage with a true accept rate at a fixed false accept rate.

A single accuracy on 6,000 pairs has a ceiling that arrives fast. One wrong pair is 0.017 percentage points; the difference between the 2015 and 2019 results in the figure below is 0.20 points, or about twelve pairs. Once the best systems disagree with the labels on a dozen pairs, label errors and the choice of pairs matter as much as the model.

03Which models took LFW from 99 percent to saturation?

Three papers mark the climb, each reporting its own LFW figure. DeepID2 (Sun, Wang and Tang, 2014) trained a convolutional network with joint identification and verification signals and reported "99.15% face verification accuracy" on LFW, cutting the error of the best previous deep result by 67 percent (arXiv 1406.4773). FaceNet (Schroff, Kalenichenko and Philbin, CVPR 2015) learned a 128-byte embedding with a triplet loss and reported "a new record accuracy of 99.63%", a 30 percent cut in error against the best published result, trained on 100 million to 200 million private face thumbnails of about 8 million identities (arXiv 1503.03832). ArcFace (Deng, Guo, Xue and Zafeiriou, CVPR 2019) added an angular margin to the softmax loss and reported 99.83 percent with a ResNet100 trained on MS1MV2, a cleaned set of 85,000 identities and 5.8 million images (arXiv 1801.07698).

The authors of CASIA-WebFace described the situation in 2014: groups with private large-scale training sets were already reaching "97% to 99%" on LFW while no large public training set existed, which is why they released 10,575 subjects and 494,414 images (arXiv 1411.7923). Their line that "data is more important than algorithm" summarised what drove the climb: each step in the figure came with a larger training set, not only a new loss.

04Why did LFW saturate, and what does saturation mean?

A benchmark saturates when the best systems score so close to its ceiling that it no longer separates them. On LFW the whole gain from 2014 to 2019 was 0.68 points, and the remaining 0.17 points to a perfect score sit within the range where pair selection and label noise decide the result. ArcFace's authors reported LFW alongside harder sets for exactly this reason: on IJB-C, a set of 3,531 subjects with 21,294 images and 117,542 video frames, the same model's result is quoted as a true accept rate of 95.6 percent at a false accept rate of one in ten thousand, a number with room left in it.

The pattern repeated across the field's benchmarks. MegaFace, which tested recognition against one million distractor photographs, was decommissioned by its authors after systems reached "benchmark performance of over 99%" (MegaFace). Detection benchmarks show the same curve: GenImage's same-generator detection accuracies above 98.5 percent are a ceiling of the same kind, and the generated-image detection benchmarks page reads them the same way, as a reason to look at the off-diagonal cells.

05What replaced LFW?

Three kinds of benchmark took over the role. Harder pair sets kept the LFW protocol but changed the pairs: the InsightFace README lists CALFW (cross-age) and CPLFW (cross-pose) with the same 5,749 identities, 13,233 images and 6K pairs, drawn so that the matched pairs differ in age or pose. Larger template sets changed the metric: IJB-B and IJB-C report true accept rate at fixed false accept rates, and Glint360K's authors report 97.2 percent TAR on IJB-C at a FAR of one in ten thousand (arXiv 2010.05222). Operational evaluation changed the data: NIST's FRTE reports FNMR, the proportion of mated comparisons below a threshold set to a specified false match rate, on visa, mugshot and border images that no paper can download.

Training sets grew in step. CASIA-WebFace's 494,414 images gave way to MS1MV2's 5.8 million, Glint360K's 17 million over 360,000 identities, and WebFace42M's 42 million faces of 2 million identities, and several of these were later withdrawn over consent. LFW itself remains a reference row in recognition papers, reported to two decimal places and read as a floor rather than a result.

06How should an LFW number be read today?

An LFW accuracy above 99.5 percent says that a model is a working deep face recogniser; it does not rank it against another model above 99.5 percent. A paper that reports only LFW has reported a saturated benchmark, and the informative numbers are the ones on IJB-C, on a cross-pose or cross-age pair set, or in a NIST report. The glossary of detection and recognition terms has saturation, TAR at FAR and FNMR as entries, and the AI image detection explainer shows the same lesson from the detection side, where a 99.9 percent same-generator score tells a reader nothing about the next generator.