Presentation attack detection: how ISO/IEC 30107 tests and reports it

This page belongs to the deepfake detection hub and goes one level below the liveness detection page, which defines the task and the two error rates. Here the subject is the standard itself: its four parts, its taxonomy of attacks, its counting rules and how a table built to it is read. Recorded-video detection, the other side of this site, is the subject of how deepfake detection works.

01What are the four parts of ISO/IEC 30107?

Part 1 is the framework. Its scope statement says it "establishes terms and definitions that are useful in the specification, characterization and evaluation of presentation attack detection (PAD) methods", and rules out "standardization of specific PAD detection methods" and "detailed information about countermeasures (i.e. anti-spoofing mechanisms), algorithms or sensors" (ISO/IEC 30107-1:2023, publicly available text). Its introduction describes the other three parts. Part 2 defines data formats for recording which PAD approach was used and what it found. Part 3 "establishes principles and methods for performance assessment of PAD mechanisms". Part 4 gives "requirements for assessing the performance of PAD mechanisms on mobile devices with local biometric recognition".

Part 3 is the one evaluators cite. Its catalogue entry records Edition 2, published January 2023, 39 pages, replacing the 2017 first edition (ISO/IEC 30107-3:2023). ISO/IEC 19795-1:2021, the general biometric performance testing standard, excludes from its scope the error rates of "people deliberately trying to subvert the intended operation of the biometric system (e.g. by presentation attacks)" and leaves them to this series.

02What does Part 1 classify?

The framework fixes the vocabulary that Part 3 then counts. A presentation attack instrument, or PAI, is a "biometric characteristic or object used in a biometric presentation attack". An artefact is an "artificial object or representation presenting a copy of biometric characteristics or synthetic biometric patterns". Liveness detection is defined separately, with the note that "Liveness detection methods are a subset of presentation attack detection methods", which is why the standard is named for attacks rather than for liveness. The scope also sets the boundary that matters most for deepfakes: the attacks considered "are those that take place at the capture device during the presentation".

Part 1 then sorts PAIs into artificial and human classes. Its Table 1 gives examples: artificial and complete, "gummy finger, video of face"; artificial and partial, "glue on finger, sunglasses, artificial/patterned contact lens, non-permanent make up"; human and lifeless, altered, non-conformant, coerced, or conformant, the last a "zero effort impostor attempt". Its Table 2 lists the detection families a PAD subsystem may use, from artefact and liveness detection to alteration, non-conformance, coercion and obscuration detection and system-level checks. A face video played on a screen is therefore an artificial, complete PAI, and a screen-replay test measures how often that species is accepted.

03How does Part 3 count the errors?

Part 3 defines the two classification error rates. Quoted from the standard in the SC37 working group slides by Christoph Busch, APCER is the "proportion of attack presentations using the same PAI species incorrectly classified as bona fide presentations in a specific scenario", and BPCER is the "proportion of bona fide presentations incorrectly classified as attack presentations in a specific scenario" (Busch, ISO/IEC 30107 slides, July 2024). The wording "using the same PAI species" is the counting rule. A PAI species is a "class of presentation attack instruments created using a common production method", so a printed photograph, a phone-screen replay and a silicone mask are three species, each with its own APCER.

Two more terms shape a report. The scenario is the fixed set of conditions, device and subjects under which the rates were measured; a rate without its scenario is not a result under the standard. Attack potential is the "measure of the capability to attack a TOE given the attacker's knowledge, proficiency, resources and motivation", where the target of evaluation is the Common Criteria term for the product under test. A report states which species were tried and at what potential, so a low APCER against cheap prints says nothing about expensive masks.

04How is a PAD result reported?

The standard does not want a single pooled error figure. The slides quote its instruction in capitals: "DON'T use neither the equal error rate (EER) nor the half-total error rate (HTER)". Instead a system is reported at a fixed operating point, as BPCER at a stated APCER. The convention "One may report BPCER when APCERAP is 5% as BPCER20" gives the naming: BPCER20 is the bona fide rejection rate at the threshold where 5 percent of attacks pass, BPCER10 at 10 percent and BPCER100 at 1 percent. The full trade-off is drawn as a detection error trade-off curve, which "reports operating points for various thresholds showing security measures versus convenience measures".

The 2023 revision added a full-system figure. IAPAR, the impostor attack presentation accept rate, is defined "in a full-system evaluation of a verification system" as the proportion of impostor attack presentations of one PAI species "that result in accept". It measures the PAD subsystem and the comparison subsystem together: an attack only counts if PAD passes it and the comparator matches it.

05How do you read a PAD table?

A table built to Part 3 has one row per PAI species and one BPCER figure for the bona fide set. Reading it means asking four questions, set out in the table below: which species were tested, what scenario produced the rates, at what operating point BPCER is quoted, and whether the figure is a PAD-only rate or a full-system IAPAR.

The public face PAD datasets are organised the same way. OULU-NPU (FG 2017) "consists of 4950 real access and attack videos" of 55 subjects, "recorded using the front cameras of six mobile devices" in three sessions, with print and video-replay attacks from two printers and two displays (OULU-NPU project page). Its four protocols are each a scenario in the standard's sense: Protocol I tests "previously unseen environmental conditions", Protocol II introduces "a previously unseen print and video-replay attack in the test set", Protocol III is "a Leave One Camera Out (LOCO) protocol", and Protocol IV combines all three. A Protocol IV result is not comparable to a Protocol I result, as a cross-dataset AUC is not comparable to an in-dataset one in why detectors fail in the wild.

06Where PAD meets deepfakes

The scope boundary in Part 1 decides which deepfake threats PAD covers. A deepfake played on a screen in front of a camera is a presentation attack, an artificial complete PAI, and APCER for that species is the right measure. A deepfake injected into the video stream after the camera does not "take place at the capture device during the presentation", so it is outside the series; the liveness and injection attacks page explains how deployments handle it. Whether a person can tell a generated face at a glance is measured by forced-choice experiments, not APCER; the Which Face Is Real game and its studies page covers those.

The quotable caveat: an APCER is a statement about one PAI species in one scenario, and no PAD figure predicts the rate for a species that was not tried.

What a conformance letter against this standard does and does not state is set out on the certified liveness detection page.