MS-Celeb-1M taken down, MegaFace retired: face dataset withdrawals

This page is part of the face recognition section. It follows the face datasets and consent page, which covers how the large sets were collected, and asks the narrower question of what a withdrawal actually changes.

01What was MS-Celeb-1M?

MS-Celeb-1M was released by Microsoft Research in 2016 as a benchmark "to recognize one million celebrities from their face images", with a training set "which contains 10M images in version 1" that the authors called the largest publicly available (arXiv 1607.08221). The images were retrieved from web search engines, about 100 per name. Two assets were distributed: images for about 100,000 people, and a target list of one million names that other groups were expected to use to gather more images themselves, as the Exposing.ai project documents (exposing.ai/msceleb).

The list was built around the word celebrity, but Exposing.ai's keyword filter on the attached biographical data found 69,219 face images of 798 people described as journalists, and the project publishes that list. The licence restricted the set to non-commercial research. The same project's citation map records commercial, academic and military affiliations among the papers that used it, which is the finding that prompted the reporting in 2019.

02Why was MS-Celeb-1M taken down?

The msceleb.org website was taken down in April 2019, days after an investigation by the Financial Times based on the Exposing.ai analysis. The audit by Peng, Mathur and Narayanan records the reason the creators gave to a press inquiry in June 2019: the dataset was removed "because the research challenge is over" (arXiv 2108.02922). The authors treat the removal as a retraction because it "responded to ethical concerns", while noting that no official statement ever said the data should no longer be used, and that the licence is no longer officially available.

Exposing.ai adds a detail about the timing: the full dataset remained downloadable from Microsoft's research data portal until at least 6 June 2019, after the takedown had been reported. Within weeks the files reappeared on Academic Torrents, seeded mostly from one company and downloaded without restriction.

03What happened to the copies?

The audit analysed 946 papers, 276 of them citing MS-Celeb-1M or a derivative, and summarised the dataset's status after the takedown in its Table 2. The original was still available through Academic Torrents and Archive.org. Five derived datasets with images from the original remained available, and 20 GitHub repositories held models pre-trained on it. In a 20 percent sample of papers published in 2020, the year after the takedown, MS-Celeb-1M or its derivatives were used 54 times, and the authors state that all their counts are lower bounds.

The paper's conclusion is that retraction "has a limited effect on mitigating harms", because the data had already run away through many sources and because "the dataset creators did not clearly state that the datasets should no longer be used". The derivatives are where most of the continued use sits. The InsightFace dataset README still lists MS1M-ArcFace at 85,000 identities and 5.8 million images, and the ArcFace journal paper reports its headline models trained on MS1MV2 and MS1MV3, both cleaned versions of the same set. Every face embedding those models produce was shaped by it, as the page on how an embedding is trained from a set like this explains.

04Why was MegaFace decommissioned?

MegaFace, from the University of Washington, was an identification benchmark with a gallery of "1 Million Photos, 690,572 Unique Users" as distractors and a training set of "4.7 Million Photos, 672,057 Unique Identities" (megaface.cs.washington.edu). Its page now carries a notice: "The MegaFace challenge has concluded, reaching a benchmark performance of over 99%", and because its goals were met and maintenance "would require considerable administrative effort, MegaFace is being decommissioned and MegaFace data are no longer being distributed". The notice gives no date.

The stated reason is different from MS-Celeb-1M's, and the effect is the same: new applicants cannot get the data, and existing copies are not recalled. The InsightFace README still lists the MegaFace training set at 672,000 identities and 4.7 million images, and MegaFace scores remain the comparison column in recognition papers published after the retirement. What the benchmark measured and the 35 to 75 percent results it produced are on the MegaFace benchmark page.

05What followed in other datasets?

The audit names DukeMTMC, VGGFace2 and Brainwash as retracted after the same investigation that exposed MS-Celeb-1M, and TinyImages after a separate ethics study. NIST discontinued distribution of all three IJB challenges on 14 March 2023, so the IJB-C column that recognition tables report, including the ArcFace TAR of 0.956 at a FAR of one in ten thousand, now rests on copies held by groups that applied before that date. The regulator decisions that applied the consent question to a commercial database, with fines of GBP 7.5 million and EUR 20 million, are on the Clearview AI rulings page listed below.

06What does a withdrawal change for a reader of benchmarks?

Three things, none of them the published number. First, a result on a withdrawn set can no longer be reproduced from the official source, so the audit's recommendation that creators state clearly whether a set may still be used matters to anyone re-running a table. Second, the demographic makeup of the training copies is fixed as it was in 2016; NIST's measurement of demographic differentials across 189 algorithms is one way to see what that produced downstream. Third, a model's training set is not visible from its output: an embedding from a network trained on MS1MV2 looks like any other, which is why the audit asks for stewardship across the whole life cycle of a dataset rather than a takedown at the end.

The deepfake datasets on this site were built the same way, from YouTube clips of public figures, and the FaceForensics++ and Celeb-DF authors distribute them by form under terms of use. The question of what a form can and cannot recall applies to them equally, and the checks on the checking a face video yourself page work from the video itself rather than from any claim about what trained the detector.