CASIA-WebFace: the 2014 face set that opened web-scale training

This page belongs to the face recognition section and gives the set its own record. Its test partner is the LFW dataset: the paper built CASIA-WebFace so that models could be trained on outside data and still be scored on LFW with the subjects kept apart.

01What is CASIA-WebFace?

CASIA-WebFace is a training set of face images with one identity label per image, built for learning a face representation from scratch. The authors give its final size as "10,575 subjects and 494,414 face images", which is about 47 images per subject. The paper's motivation was a gap in public data. Groups with private data had reached 97 percent to 99 percent on LFW, and in the authors' words "none of large scale face dataset is publicly available".

The set also changed how results were reported. Before it, the strongest LFW numbers came from private training sets, so two papers could differ in data more than in method. The authors wrote that "the current situation in the field of face recognition is that data is more important than algorithm", and they asked others to train on CASIA-WebFace and report on LFW.

02How was it collected from IMDb?

The authors used IMDb because every celebrity there has a page with a name, a "main photo" and a "photo gallery". They searched for celebrities born between 1940 and 2014, crawled their names and pages, and skipped any celebrity without a main photo. That gave 38,423 subjects and 903,304 images.

A multi-view face detector then ran over every image. It left 844,126 images and found 1,556,352 faces. Many photos appear in the galleries of several celebrities at once, so the paper notes that the true numbers of images and faces are smaller than the raw totals. Detection came first because a gallery photo often holds several faces and only one of them belongs to the page it was found on.

03How were identities assigned and cleaned?

A face found on a celebrity's page does not always belong to that celebrity. The authors used a tag-and-similarity clustering method with five steps. A pretrained recognition engine extracts a feature template for each face; the main photo of each celebrity serves as the seed; single-face images augment the seed set; faces in the remaining gallery photos are matched to celebrities by similarity, constrained by the name tags on each photo; and faces are cropped into one folder per celebrity and checked by hand to delete false groups.

Two filters followed. Subjects with fewer than 15 face images were removed. Then the authors compared names with LFW and found that 1,043 subjects share names with LFW subjects; those were removed too. The paper adds that a small number of wrongly labelled images remain and argues that they do not harm training. After cleaning, the final count was 10,575 subjects and 494,414 images.

04Why remove 1,043 subjects that also appear in LFW?

LFW is the test set, and a person who is in both sets would let a model memorise the test identity during training. The authors checked duplicates by the edit distance between names, removed the 1,043 matches, and described the result as a training set that is kept apart from LFW. Combining CASIA-WebFace with LFW gave, in their words, "a new benchmark for large scale face recognition in the wild".

The check is by name, so it has the limits of names. Two different people can share a name, and one person can appear under two spellings. Later benchmarks took the same precaution with their own wording. NIST's IJB-C readme says its database "is designed to have no overlap with the popular face recognition benchmarks, such as University of Oxfords VGG-Face dataset and the CASIA WebFace dataset", which is the same idea applied in the other direction.

05How big was it next to the other sets of 2014?

The paper's Table 1 lists six face sets with their sizes and whether they were available. CASIA-WebFace was about 37 times the size of LFW in images, and it was the largest public set in the list.

The authors write that its size "rank second in the literature", behind the private SFC set used by Facebook. SFC had nearly nine times the images and fewer than half as many subjects. CASIA-WebFace was the largest public set in the table. The face recognition datasets compared page carries it forward against the later sets of millions of images.

06What did it score on LFW?

The authors trained an 11-layer convolutional network and tested it on LFW. Their single-network scheme reached 97.73 percent under the unrestricted protocol, which the paper sets against 97.35 percent for the seven-network ensemble of DeepFace and 97.75 percent for the four-network ensemble of DeepID2. A single network matching ensembles trained on private data was the paper's evidence that the dataset was good enough.

The paper also reported results on YouTube Faces and under the BLUFR protocol, which uses all 13,233 LFW images and measures verification and open-set identification at low false accept rates. LFW itself stopped separating systems years later, as the LFW dataset page shows with the published accuracies from 2014 to 2019.

07Why do papers still train on it?

It is small enough to train on one machine and old enough that every result on it has a reference. The ArcFace paper's loss comparison trains a ResNet50 on CASIA and reports an accuracy of 99.53 percent on LFW for its angular margin loss with m at 0.5, so a researcher can compare a new loss to that row (arXiv 1801.07698). The ArcFace page explains the loss. InsightFace's datasets README still lists it, as "CASIA-Webface (10K ids/0.5M images)".

Size shows in harder tests. The Partial FC paper's Table 1 trains the same IR50 backbone on different sets and scores each on IJB-C at a false accept rate of 10^-4. Trained on CASIA the true accept rate is 87.2 percent; on MS1MV2 it is 96.1 percent; on Glint360K it is 97.1 percent (arXiv 2010.05222). A good score on LFW with CASIA-WebFace therefore does not predict a good score on a harder benchmark. OpenFace, an open recognition model released in 2015, trained on 500,000 images that combined CASIA-WebFace and FaceScrub, and the InsightFace page covers a toolkit whose README names CASIA-Webface among its training data.

08What does the paper say about the people shown?

The paper describes a crawl of one website: celebrities' names, main photos and gallery photos. It does not describe consent from the people shown, and it does not state terms of use for the images; a text search of the full paper finds neither. Treat the set as a collection of web photographs of public figures whose rights position the paper leaves open, the same position as other sets built by crawling. The face datasets and consent page gathers the withdrawals and rulings that followed this kind of collection.