Is face recognition AI? What the models are and are not

This page belongs to the face recognition section. It answers the definition question; the step-by-step version is on how face recognition works.

01What does it mean to call face recognition AI?

Artificial intelligence is a broad label, and face recognition qualifies under the narrow, technical sense that matters here: the system's behaviour comes from parameters learned from data, not from rules a programmer wrote. The DeepFace network of 2014 "involves more than 120 million parameters". Nobody wrote a rule for what makes two faces the same person; the parameters were adjusted until faces of one identity landed close together in the model's output space.

The label does not mean the model understands faces, reasons about people or decides anything. It maps an image to numbers. A person, a threshold or a larger system decides what to do with the distance between two sets of numbers.

02How did face recognition work before deep learning?

The 1991 Eigenfaces system by Turk and Pentland was an early landmark. Its abstract describes "projecting face images onto a feature space that spans the significant variations among known face images", and says those features "are the eigenvectors (principal components) of the set of faces; they do not necessarily correspond to features such as eyes, ears, and noses." That is a learned, linear representation fitted to a set of faces, with no network and a single projection.

By 2014 the common approach was different. The DeepFace paper (Taigman, Yang, Ranzato and Wolf, CVPR 2014) states that "Most current face verification methods use hand-crafted features", and that leading systems "employ tens of thousands of image descriptors". The authors argue that "learningbased methods have started to outperform engineered features, because they can discover and optimize features for the specific task at hand". Their own network was trained on "an identity labeled dataset of four million facial images belonging to more than 4,000 identities" and reached 97.35% on LFW.

03What does a modern model actually learn?

A modern model learns an embedding: a function from a cropped, aligned face image to a fixed-length vector. FaceNet (Schroff, Kalenichenko and Philbin, 2015) describes it as a system that "directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity". It trained on between 100M and 200M face thumbnails of about 8M identities, a private set.

ArcFace (Deng, Guo, Xue and Zafeiriou, CVPR 2019) states that "Face representation using Deep Convolutional Neural Network (DCNN) embedding is the method of choice" for large-scale recognition. Its setup is typical: 112 by 112 face crops generated from five facial points, a ResNet-50 or ResNet-100 network, and a 512-dimensional embedding at the end. Its main training set, MS1MV2, is "a semi-automatic refined version" of MS-Celeb-1M, with 85,000 identities and 5.8 million images. The loss adds an angular margin so that embeddings of one identity cluster tightly; the ArcFace page explains it, and the embeddings page shows how the vectors are compared.

04What is a face recognition model not?

It is not a rule set, and not a lookup of a stored photograph. At match time a system compares vectors that a trained network produced from images.

It is not a generator. A face recognition network scores or compares faces; it does not create them. Some face generators borrow one: SimSwap uses a pretrained ArcFace model for the identity vector, and FaceShifter uses a pretrained recognition model as its identity encoder. The same field's detectors share the pipeline without sharing the task, as the bridge from recognition to deepfake detection explains.

It is not an identity oracle. A verification system returns a similarity that a threshold turns into accept or reject, and an identification system returns the nearest entries in a gallery. Both can be wrong, and the error rates depend on the threshold.

05Where does a recognition model's behaviour come from?

From its training data and its loss. The model can only separate the kinds of faces and conditions it has seen, which is why dataset choice is a research question and a legal one. WebFace260M collected 260 million faces of 4 million identities from the internet before cleaning to 42 million faces of 2 million identities, and the datasets and consent page records the consent and licence position of such sets.

Training data also sets what the benchmarks reward. The AI face recognition page traces how web-scale sets saturated LFW and moved evaluation to harder protocols. A model's score on one benchmark describes that benchmark, and a better score on a larger training set is a finding about the set as well as the model.