GenConViT: the open-source deepfake detector that led the 2024 in-the-wild test

This page is part of the deepfake detection hub. It is the model-level companion to the deepfake detection in the wild page, which covers the benchmark that produced the 0.63.

01What is GenConViT?

GenConViT stands for Generative Convolutional Vision Transformer. It was published by a group at Ghent University's IDLab and Addis Ababa University, first posted in July 2023 and revised in March 2025 (arXiv 2307.07036). The abstract states the design in one sentence: the model "combines ConvNeXt and Swin Transformer models for feature extraction, and it utilizes Autoencoder and Variational Autoencoder to learn from the latent data distribution".

The point of the generative parts is to look at two things at once. The convolutional and transformer branches learn visual artefacts in the face crop; the autoencoder and variational autoencoder learn what a reconstruction of that crop looks like, so that a face the model cannot reconstruct cleanly carries a second signal. The code is public at the authors' repository, and the paper notes it was used by TrueMedia.org, a non-profit, for fact-checking support.

02How is the model built?

GenConViT is two networks whose outputs are averaged. Network A holds an autoencoder; network B holds a variational autoencoder. Each feeds a hybrid ConvNeXt-Swin block that takes a 224 by 224 RGB face crop together with the latent feature from its generative part. The authors used the "tiny" variants of both backbones, convnext_tiny and swin_tiny_patch4_window7_224, pretrained on ImageNet-1k, and trained with Adam at a learning rate of 0.0001 with a batch size of 32 for network A and 16 for network B.

Training data came from five sets: DFDC, FaceForensics++ (c23 and c40), the TM set of 4,380 fake and 2,563 real videos used for training only, DeepfakeTIMIT and Celeb-DF v2. About 30 frames were extracted per training video, with more from real videos to offset the 6 to 1 fake to real ratio in DFDC. For testing, 3,972 videos were held out and 15 frames per video were scored and averaged into one prediction. The pipeline is the standard one that every face model shares, detect, crop, resize, classify, and the face recognition technology overview describes the recognition version of it.

03What does the GenConViT paper report?

Table IV of the paper gives accuracy on the held-out test videos: 98.50 percent on DFDC, 97.00 percent on FaceForensics++, 98.28 percent on DeepfakeTIMIT and 90.94 percent on Celeb-DF v2, an average of 96.05 percent. Table V gives AUC of 99.9 percent on DFDC, 99.6 percent on FaceForensics++ and 98.1 percent on Celeb-DF v2, and the text summarises the model as reaching "an average accuracy of 95.8% and an AUC value of 99.3% across the tested datasets". The Celeb-DF v2 row is the weak one: 83.00 percent on real videos against 98.80 percent on fakes, so the errors there are false alarms.

The authors add a caveat that applies to every comparison table in this field: "the testing sub-datasets of DFDC and FF++ are usually not shared in the scientific literature. Hence, the exact evaluated datasets may differ". Two papers quoting accuracy on "DFDC" may have tested on different clips. The published detector results by test set table records test split and protocol alongside each number for that reason.

04What happened on Deepfake-Eval-2024?

Deepfake-Eval-2024 collected 45 hours of video, 56.5 hours of audio and 1,975 images that circulated on 88 websites in 52 languages during 2024, and ran open-source and commercial detectors on them (Chandra et al., arXiv 2503.02857). Table 3 reports each open-source model's AUC on the new set beside its AUC on its original publication test data, re-run by the benchmark authors. GenConViT scored 0.63 on Deepfake-Eval-2024 against 0.96 on its original data, with precision 0.60 and recall 0.50 on the new set. The other two video models, FTCN and Styleflow, scored 0.50 and 0.51, which the paper calls "the same as random guessing". Across all video models the average AUC drop was 50 percent.

The benchmark's error analysis names one cause. Off-the-shelf GenConViT and FTCN had "an average 21.3% lower accuracy on videos which appear to be generated by a diffusion model", generators that did not exist when the training sets were made. The 0.63 therefore reads two ways: the best open-source video result in that study, and a model that fails on the newest class of fake.

05Does fine-tuning repair the gap?

Partly, and under a different protocol. The benchmark authors fine-tuned every open-source model on 60 percent of Deepfake-Eval-2024 and tested on the remaining 40 percent. GenConViT rose to 0.82 AUC and 0.75 accuracy (Table 4), the highest of the video models and close to the best anonymous commercial video model's 0.79 AUC and 0.78 accuracy (Table 5). FTCN reached 0.71; Styleflow 0.56. After fine-tuning, the accuracy gap on diffusion-generated videos narrowed from 21.3 percent to 5.4 percent.

The fine-tuned figure is not an off-the-shelf figure. A model that has trained on part of a distribution is being tested on the rest of it, and the paper reports the two tables separately. The authors' summary is that even fine-tuned models, with "the peak accuracy reaching 0.75 for videos", stay below their 90 percent lower-bound estimate for human deepfake forensic analysts.

06Why does a 99 percent model score 63 percent?

GenConViT's own paper already contained the answer. In an ablation the authors trained on four of their five datasets and held out the fifth: with Celeb-DF v2 held out, accuracy on Celeb-DF v2 fell to 55.67 percent, with 11.56 percent on its fake videos; with DFDC held out, 62.08 percent overall and 28.84 percent on fakes. They conclude that "deeper or wider CNN architectures alone do not necessarily guarantee improved robustness to domain shifts". Deepfake-Eval-2024 is that held-out experiment run against the whole internet, and its 0.63 sits between the paper's own held-out figures.

The error type matches as well. In the ablation, held-out fakes were called real; on Deepfake-Eval-2024 the recall was 0.50, meaning half the fakes passed. This is the same shape as the benchmark-to-field gap in face recognition, where the NISTIR 8280 report found error rates on operational images that no academic pair list had exposed, and where the FMR, FNMR, FPIR and FNIR rates were defined so that both kinds of error are visible at one threshold. No single benchmark score predicts how a detector performs on manipulations it has not seen.

Other transformer detectors (TALL, CLIP-based, VideoMAE) are compared on the transformer detectors in DeepfakeBench page.