Deepfake audio detection: methods, metrics and the published results

This page belongs to the deepfake detection hub and is the methods companion to the audio deepfake benchmarks page, which lists the datasets. The deepfake voice page covers how cloned speech is made.

01What does a deepfake audio detector look at?

A speech countermeasure takes a recording and returns one score. The ASVspoof 2021 paper names the four ways the recording can be fake: impersonation, voice conversion, text-to-speech synthesis and replay (arXiv 2210.02437). The first three leave traces in the signal that the generator or converter introduced; replay leaves traces of the loudspeaker, the room and the second microphone.

Detectors differ in what they feed the classifier. The classical route computes a hand-designed spectral feature, such as constant Q cepstral coefficients or linear frequency cepstral coefficients, and models it with a Gaussian mixture model or a convolutional network. The more recent route feeds the raw waveform to a network that learns its own filters. ASVspoof 2021 shipped both as baselines: B01 and B02 used CQCC and LFCC with GMMs, B03 used LFCC with an LCNN-LSTM, and B04 fed the raw waveform to RawNet2.

02How are countermeasures scored?

Two metrics carry the field. The equal error rate is the operating point at which the share of bona fide recordings rejected equals the share of spoofed recordings accepted. It describes the countermeasure on its own and is the primary metric for ASVspoof 2021's deepfake task, because, as the paper states, "the DF task does not involve the use of an ASV system". The metrics page of this section places EER next to AUC and the biometric error rates.

The tandem detection cost function describes the countermeasure as a gate in front of a speaker verification system. It weights the countermeasure's misses and false alarms by what they cost the verification system behind it, so a detector that blocks genuine speakers is penalised as well as one that lets fakes through. The ASVspoof 2021 paper uses min t-DCF as the primary metric for the logical access and physical access tasks, the two in which a verification system sits downstream. Lower is better for both metrics.

03What did ASVspoof 2021 find?

The 2021 edition drew 54 teams across three tasks. In the logical access task the top submission, T23, reached a min t-DCF of 0.2176 and an EER of 1.32 percent on the evaluation set, against 0.3445 and 9.26 percent for the best baseline, B03 (Table VIII of arXiv 2210.02437). T23 fused several networks, LightCNN, ResNet and LSTM, over raw waveform and raw spectrogram inputs, with transmission codec augmentation during training, according to the paper's Table V.

The deepfake task was harder. Its evaluation set held 14,869 bona fide and 519,059 spoofed trials generated with more than 100 spoofing algorithms, then compressed with mp3, m4a and ogg codecs at low and high bitrates. The best system, again T23, scored an EER of 15.64 percent; the baselines B04 and B03 scored 22.38 and 23.48 percent. The paper notes that 23 of 33 systems were under 10 percent EER on the progress subset but all were above 15 percent on the evaluation subset, which is the generalisation gap in one sentence.

Both post-challenge figures come from the same Table VIII and are discussed in the next section.

04Which architectures lead now?

The paper identifies two trends in the systems that followed the challenge: end-to-end networks that work on the raw waveform, and heavy data augmentation. RawNet2, the B04 baseline, was the first raw-waveform network applied to anti-spoofing; its authors reported the second-best published result on the A17 attack of ASVspoof 2019 and noted that raw audio "has potential to learn cues that are not detectable using more traditional countermeasure solutions" (arXiv 2011.01108).

AASIST built on RawNet and graph attention networks to model spectral and temporal artefacts in one system without score-level ensembles; its authors reported a 20 percent relative improvement over the previous state of the art, and a light variant with 85K parameters that still beat the competing systems (arXiv 2110.01200). The ASVspoof 2021 paper calls AASIST "representative of progress in the use of E2E architectures".

The best published numbers on the 2021 data replaced the RawNet front end with a self-supervised speech model such as wav2vec, reaching 0.2066 min t-DCF and 0.82 percent EER on logical access and 2.85 percent EER on the deepfake task. The paper adds a condition that matters when reading those rows: models pre-trained on external data are "not compliant with the ASVspoof training protocol", so the post-challenge figures and the challenge figures are not the same contest.

05Why do results fall on found recordings?

Every ASVspoof figure is measured on speech the organisers synthesised from a known corpus. Müller et al. built a set of 37.9 hours of found recordings of celebrities and politicians, 17.2 hours of them deepfakes, and reported a "performance degradation of up to one thousand percent" when detectors trained on ASVspoof data met it (arXiv 2203.16263). The same study found that the choice of input feature moved results a long way: constant Q or log spectrograms instead of mel spectrograms improved EER by 37 percent on average.

Deepfake-Eval-2024 measured the same gap for the current generation with 56.5 hours of audio collected from social media in 2024 and reported that open-source audio detectors lost 48 percent of their AUC against their original benchmarks (arXiv 2503.02857). The in-the-wild face results page gives the video and image figures from the same study, and they move the same way.

06How to read an audio detection score

A published EER or t-DCF holds for the evaluation set, the codec conditions and the training protocol it was measured under, and nothing else. The ASVspoof 2021 abstract puts the caveat in the authors' own words: detection solutions "offer some resilience to compression effects" but "lack generalization across different source datasets". A score of 1.32 percent EER on ASVspoof 2021 logical access does not predict the error rate on a phone call recorded this year with a generator released last month, which is the setting of the documented deepfake scams that used cloned voices.

Three conditions should travel with any figure quoted from this page: which task it comes from, because LA, PA and DF are different problems; whether the system followed the challenge training rules; and whether the test speech was synthesised by the organisers or found online. A detector score is an inference from the waveform, and the provenance versus detection page explains why a signed record of a recording's origin outranks it when one exists. ASVspoof 5, the next edition, moved the deepfake track to minDCF as its primary metric, so figures from 2024 onward are not directly comparable with the 2021 EERs either.