The ICCV 2021 masked face recognition challenge, WebFace260M track

This page describes the challenge from its public report (arXiv 2108.07189), because the address it sits at was the challenge's own page from 2021 to 2025 and is still cited by package pages, company blogs and researcher homepages. It is part of this site's face recognition benchmarks section; the companion page covers the WebFace260M dataset the track was built on.

01What the challenge was

In 2021 almost everyone in public wore a mask, and most face recognition systems had been built and tested on unoccluded faces, where the eyes, nose and mouth are all visible. Asking a traveller to remove a mask at an airport gate or a lab door meant exposure risk. The challenge organisers' report puts the problem plainly: there was no publicly available real-world masked face recognition benchmark, because the need had arrived suddenly.

The ICCV 2021 Masked Face Recognition Challenge (MFR) was the response. It had more than one track; this page covers the WebFace260M track, which the WebFace260M group organised and which used their dataset and their FRUITS protocol (Face Recognition Under Inference Time conStraint). The aim stated in the report was to "push the frontiers of practical" masked face recognition, with the word practical doing real work: entries were judged under a time limit, not only on accuracy.

02The two test sets and the protocol

The standard set

The report says public evaluation sets were mostly saturated or contained noise, so the organisers gathered a new one: 2,478 celebrities and 60,926 faces, described in the report as an elaborated test set. Entries were evaluated on it in the usual 1:1 verification style, where the system is asked whether two images show the same person.

The masked set

The second set was what made the challenge new: the organisers describe it as the world's largest real-world masked test set. The phrase real-world matters, because a mask pasted onto an unmasked photo is a different, easier problem than the lighting, fit and shadow of a mask someone is actually wearing.

The time limit

Under FRUITS, a submission had to complete its work within a fixed time budget. That rule carried over from the WebFace260M paper, which argued that accuracy alone misleads when a slightly better model is several times slower. In a challenge setting it also levelled the field between teams with large compute and teams without it, since a huge ensemble that could not finish in time scored nothing.

03How entries were scored

Face verification benchmarks score a system by its true accept rate at a fixed false accept rate, so it is rewarded for recognising the same person while held to a strict limit on how often it matches two different people. Read the report's result tables with that format in mind: the operating point is part of the number. The face verification metrics page explains the format, and why a single accuracy percentage would say much less.

The same quantities appear under different names in deepfake detection, as true positive rate and false positive rate on an ROC curve. This site's metrics page maps one vocabulary onto the other.

04What the report found

The public report, written while the challenge was still open, gives the first-phase picture: 69 teams, 833 submitted solutions, and 49 teams above the organisers' baseline. A second phase ran until 1 October 2021 with a live leaderboard, and the authors said they would update the report.

This page does not reproduce the leaderboard. Rankings from a live competition belong to the organisers and change as phases close; copying a snapshot here would misdescribe a moving result. What can be said from the report is that masked recognition was tractable with large-scale training data and that dozens of teams beat the organisers' baseline within a practical time budget. Teams that placed, including industry labs, described their own results on their own pages, and some of those pages name this address as the challenge home.

05Where the challenge lives now

The challenge's data, protocol and ongoing evaluation moved to Codalab at the end of 2021, according to the news item on the old home page of this domain. The report is on arXiv, the organisers' code is on their own GitHub, and the WebFace260M dataset is requested through its authors, as described on the dataset page. This site is not the organiser, holds no submissions or results files, and cannot answer questions about entries.

06The same question for deepfakes

A masked face is a face with part of its evidence removed. A deepfake is a face with part of its evidence forged. Both problems are handled by the same pipeline (detect the face, align it, embed it, compare or score it), and both are benchmarked with the same curves. The bridge page follows that pipeline from "who is this?" to "is this real?", and the liveness and injection attacks page covers the place where the two meet in practice: identity checks that must recognise a face and decide whether it is live at the same time.