Deepfake video vs image detection: what changes between them

This page belongs to the deepfake detection hub. It compares the two kinds of detector side by side; the detail of clip-level methods sits on the page about temporal cues and clip scoring.

01What does a video detector see that an image detector cannot?

A video detector sees how a face changes from frame to frame. The FTCN paper (Zheng et al., ICCV 2021) starts from the observation that face manipulation techniques "are struggling to generate temporal coherent face videos", so inconsistency between frames is a signal an image detector cannot use (arXiv 2108.06693). FTCN keeps its convolution kernels along time and reduces the spatial kernel size to 1, then adds a temporal transformer for long-range coherence.

An image detector has only the pixels of one frame. It can look for blending boundaries, frequency artefacts or traces left by a generator, but not for flicker or motion that does not fit. The cost of the temporal signal is that the detector needs a face to be found across enough frames; Deepfake-Eval-2024 notes that some models fail on files because of "requirements for a face to be detected in a certain number of frames".

02How does a clip get one score?

A clip gets one score by aggregation. Detectors that work on single frames produce a score per frame, and a clip score is made by combining them. FTCN's authors, comparing against image-based methods, "average the model predictions for each frame across the entire video" and report video-level AUC (arXiv 2108.06693). The TALL paper does the same: "we calculate video-level predictions for the image-based method and average the predictions across the entire video" (arXiv 2307.07494).

Native video models skip that step. TALL samples 8 clips of 4 frames and arranges them into one thumbnail layout that a single network scores. The unit of the decision is therefore different, and so is the metric. DeepfakeBench notes that "some are at the frame level, while others are at the video level, leading to unfair comparisons", and adopts frame-level evaluation (arXiv 2307.01426). The page on frame-level and video-level AUC shows how far the two numbers can sit apart for one detector.

03What do image detectors target?

Image detectors mostly target a different problem. The three image models in Deepfake-Eval-2024 are UFD, a single-layer classifier on a CLIP backbone, DistilDIRE, based on diffusion inversion, and NPR, a convolutional network. The paper's Table 3 note lists their original test data as including generated images from latent diffusion models, Glide, DALL-E mini, Stable Diffusion v1 and ADM. That is a different task from finding a swapped face in a video.

The paper draws the same line from the data side: "image datasets largely repurpose video frames" or "target general AI-generated content rather than deepfakes". So "image deepfake detection" covers two tasks: scoring frames cut from swap or reenactment videos, and scoring fully generated images. The page on AI-generated faces covers the second.

04How far did each fall on Deepfake-Eval-2024?

Both fell, by similar amounts. The paper reports an average AUC drop of 50 percent for video, 48 percent for audio and 45 percent for image models against each model's original benchmark (arXiv 2503.02857). Table 3 gives the six models behind the video and image figures.

A reader who averages the three per-model relative drops in this table gets about 41 percent for video and 45 percent for image. The abstract states 50 percent for video, and the table does not show how that figure was derived, so this page quotes the abstract and prints the table. The paper's own summary is that many off-the-shelf models "have an AUC close to 0.5, the same as random guessing".

05Does fine-tuning or a commercial model close the gap?

Fine-tuning helps video more than image on average, and commercial models do better in both. After fine-tuning on 60 percent of Deepfake-Eval-2024 and testing on the other 40 percent, AUC for open-source models rose by an average of 57.6 percent for video and 4.5 percent for images. GenConViT reached 0.82, FTCN 0.71 and Styleflow 0.56; among image models UFD and DistilDIRE both ended at 0.56 and NPR at 0.73. The best commercial model, tested anonymously, scored 0.79 AUC on video and 0.90 on images (Tables 4 and 5).

The authors add that commercial and fine-tuned models "do not yet reach the accuracy of deepfake forensic analysts". The Deepfake-Eval-2024 page lists the benchmark's construction, and the page on fakes found in the wild covers the earlier in-the-wild sets.

06How should a video score and an image score be compared?

They should not be compared directly. A video AUC and an image AUC answer different questions on different data, and each depends on the unit: frame or clip. Before reading a number, check four things: the modality of the test data, whether the score is per frame or per clip, which manipulations the training data covered, and whether files the model could not process were dropped. Deepfake-Eval-2024 excludes any file for which a model produced no prediction.

DeepfakeBench shows the split in one project: its README lists 36 supported detectors, 28 image and 8 video, and its published results table is frame-level AUC (README). A clip-level score from another paper is not on the same axis as that table.