Deepfake-Eval-2024: the benchmark of fakes that circulated in 2024

This page belongs to the deepfake detection hub and describes one dataset in depth; the in-the-wild benchmarks page compares it with WildDeepfake and the audio equivalent, and the datasets table lists its counts beside the curated sets.

01What is Deepfake-Eval-2024?

Deepfake-Eval-2024 is a labelled test set of real and fake media collected after it circulated, built by a team from TrueMedia.org, the University of Washington and the Allen Institute for AI (arXiv 2503.02857). Table 1 of the paper gives 45.1 hours of video at an average 576 by 720 pixels, 56.5 hours of audio at 44.66 kHz and 1,975 images at an average 1,024 by 1,024 pixels. Audio includes the soundtracks of the videos as well as audio-only clips, so the three parts overlap in source material.

The authors' stated purpose is a benchmark that matches the threat rather than the lab. They write that most prior sets "use outdated manipulation techniques" and that the most recent in-the-wild sets before theirs were published in 2022 for audio and 2023 for video, before diffusion models and commercial voice conversion spread. The dataset is distributed through the project's repository, which links a Hugging Face dataset page (GitHub).

02How was the data collected?

Every item came from a person who suspected it. The paper states that "each item in Deepfake-Eval-2024 comes from a social media or TrueMedia.org user who flagged the media as potentially AI-manipulated in 2024". TrueMedia.org was a non-profit checking platform used by journalists and fact-checkers from April 2024 and opened to the public in September 2024; users submitted a social media link or uploaded a file. The team also ran a bot on X that users could tag to submit a post, and added posts that X Community Notes had flagged as possibly manipulated.

The result is a sample of what people doubted, not of what was posted. The five most common origins were X, direct upload, TikTok, Instagram and YouTube, from 88 web domains in all. Speech recognition identified 52 languages across the set, 42 in the audio part and 49 in the video part, and 78.7 percent of the material is English. The authors estimate from a 10 percent labelled sample that the visual data is 54.6 percent white and 45.4 percent non-white by perceived race.

03How were the items labelled?

Duplicates were removed with hashes and manual review, and anything without a photorealistic face was dropped: cartoons, art and scenes without people. A team of seven, three experienced labellers and four research interns, labelled each item fake, real or unknown, using reverse image search, fact-check articles, Community Notes and forensic markers such as the match between mouth movement and sound. Items labelled unknown were excluded. Two rules are worth knowing. A face swap was labelled fake only if it was made after 2023, on the authors' reasoning that most later swaps use AI, and otherwise unknown. Audio was labelled fake only when audible traits pointed that way and at least two commercial audio detectors agreed; the paper states that "audio detectors alone are never used to label" and names the resulting correlation as a limitation of the audio part.

The team lead double-reviewed a random 10 percent. Disagreement was 6.6 percent for video, 7.9 percent for audio and 9 percent for images, and real against fake disagreement, the case that matters for a detector, was 2.1, 0.6 and 0 percent. From that the authors posit that "deepfake detection models should be able to achieve at least 90% accuracy on Deepfake-Eval-2024". Nothing they tested reached it.

04What did open-source detectors score?

Nine open-source models were run off the shelf, three per modality, chosen before testing and none dropped afterwards. Table 3 of the paper gives each model's AUC on Deepfake-Eval-2024 beside its AUC on its original test data. For video, GenConViT scored 0.63 against 0.96, FTCN 0.50 against 0.87 and Styleflow 0.51 against 0.95. For audio, AASIST scored 0.43 against 1.00, RawNet2 0.53 against 0.99 and P3 0.58 against 1.00, with clips cut to four second segments to fit the models' limits. For images, UFD scored 0.56 against 0.94, DistilDIRE 0.52 against 0.99 and NPR 0.53 against 0.98.

Averaged within each modality, that is the 50, 48 and 45 percent drop in the abstract. The paper's reading is that "many off-the-shelf models have an AUC close to 0.5, the same as random guessing", and that the models learned correlations present in academic training sets that do not hold in current circulated media. The error analysis adds one specific cause: off-the-shelf GenConViT and FTCN averaged 21.3 percent lower accuracy on videos that appeared to come from a diffusion model than on other videos. The deepfake detection models ranked page shows how different the picture is on curated sets, where the same kind of detector scores between 0.70 and 0.79 AUC on unseen data.

05Does fine-tuning or a commercial model close the gap?

Partly. The authors fine-tuned every open-source model on 60 percent of Deepfake-Eval-2024 and tested on the other 40 percent (Table 4). GenConViT rose to 0.82 AUC and 0.75 accuracy, FTCN to 0.71 and 0.65, Styleflow to 0.56 and 0.53. Audio models gained most: AASIST 0.91, RawNet2 0.88 and P3 0.92 AUC. Image models barely moved, UFD and DistilDIRE at 0.56 AUC and NPR at 0.73, which the authors attribute to too little image data to shift large models. The average AUC gains were 57.6 percent for video, 80.6 percent for audio and 4.5 percent for images, and the diffusion-video accuracy gap narrowed from 21.3 to 5.4 percent.

Twenty-two commercial models, six video, eight audio and eight image, from companies that partnered with TrueMedia.org were evaluated anonymously at their December 2024 versions, with every vendor blind to the test data. Because of per-query cost, all models ran on a subset and only the top three per modality ran on the whole set. The best commercial model per modality (Table 5) reached 0.79 AUC and 0.78 accuracy on video, 0.93 and 0.89 on audio, and 0.90 and 0.82 on images. The paper states that "no commercial models that we evaluated had an accuracy of 90% or above", the floor the labelling disagreement implies for a forensic analyst.

06Why do numbers differ between sources?

Three figures appear in more than one form. The video total is 45 hours in the paper's abstract, 45.1 hours in its Table 1 and 44 hours in the repository README; this page uses the paper's 45. The best off-the-shelf open-source AUC is printed as 0.63 for GenConViT in Table 3, while a sentence in section 4.1 of the same version gives a maximum of 0.58, which is the audio model P3's figure; this page uses the table. And the commercial video figure of 0.79 is the best model's score, while the commercial video models as a group ranged from 0.64 to 0.79, so a reader quoting "commercial detectors scored 0.79" should say "the best one did". This site's results table carries the Deepfake-Eval-2024 rows with these conditions attached.

07What are the limits of the benchmark?

No single in-the-wild score transfers to the next year's fakes, because the set is a sample of what users doubted in one year, labelled by people who could not always know. The authors list the limits themselves: audio labels depend in part on commercial detectors, items whose truth could not be established were dropped, the sample skews to English and to platforms with active fact-checking, and commercial models were compared on a subset first. Real against fake disagreement among trained labellers was under 2.5 percent, which is low, but the unknown category absorbed the hardest cases before any detector saw them.

The benchmark also says nothing about demographic error rates, although the authors did label 10 percent of the visual data by perceived race, so a later analysis is possible. The face recognition bias page explains what such a measurement has to specify before a number can be called a bias. For now, Deepfake-Eval-2024 answers one question well: in 2024, detectors trained on curated sets scored near chance on what people actually encountered.