Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jun 12;11(6):194.
doi: 10.3390/jimaging11060194.

Evaluating Features and Variations in Deepfake Videos Using the CoAtNet Model

Eman Alattas  1   2 , John Clark  2 , Arwa Al-Aama  3 , Salma Kammoun Jarraya  1
Affiliations

Evaluating Features and Variations in Deepfake Videos Using the CoAtNet Model

Eman Alattas et al. J Imaging. .

Abstract

Deepfake video detection has emerged as a critical challenge in the realm of artificial intelligence, given its implications for misinformation and digital security. This study evaluates the generalisation capabilities of the CoAtNet model-a hybrid convolution-transformer architecture-for deepfake detection across diverse datasets. Although CoAtNet has shown exceptional performance in several computer vision tasks, its potential for generalisation in cross-dataset scenarios remains underexplored. Thus, in this study, we explore CoAtNet's generalisation ability by conducting an extensive series of experiments with a focus on discovering features and variations in deepfake videos. These experiments involve training the model using various input and processing configurations, followed by evaluating its performance on widely recognised public datasets. To the best of our knowledge, our proposed approach outperforms state-of-the-art models in terms of intra-dataset performance, with an AUC between 81.4% and 99.9%. Our model also achieves outstanding results in cross-dataset evaluations, with an AUC equal to 78%. This study demonstrates that CoAtNet achieves the best AUC for both intra-dataset and cross-dataset deepfake video detection, particularly on Celeb-DF, while also showing strong performance on DFDC.

Keywords: CoAtNet; Generative Adversarial Networks (GANs); computer vision (CV); deepfake; digital multimedia forensics.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflicts of interest.

Figures

Figure 1
Figure 1
Deepfake manipulation types: (top) head puppetry, (middle) face swapping, and (bottom) lip syncing. Source: [2].
Figure 2
Figure 2
Vision Transformer (ViT) architecture. Source: [49].
Figure 3
Figure 3
CoAtNet architecture. Source: [54].
Figure 4
Figure 4
Proposed framework for evaluating the generalisation of the CoAtNet model.
Figure 5
Figure 5
FaceForensics++ dataset examples. Source: [59].
Figure 6
Figure 6
DFDC dataset examples. Source: [60].
Figure 7
Figure 7
Celeb-DF dataset examples. Source: [61].
Figure 8
Figure 8
FaceShifter dataset examples. Source: [59].
Figure 9
Figure 9
Frame selection approaches for performance improvements.
Figure 10
Figure 10
Used frame selection approaches. Frames are extracted from Celeb-DF dataset.
Figure 11
Figure 11
CoAtNet16A’s performance generalisation gap for cross-dataset evaluation.
Figure 12
Figure 12
AUC scores of CoAtNet16A model variants compared to state-of-the-art deepfake detectors.
See this image and copyright information in PMC

References

    1. Shelke N.A., Kasana S.S. A comprehensive survey on passive techniques for digital video forgery detection. Multimed. Tools Appl. 2020;80:6247–6310. doi: 10.1007/s11042-020-09974-4. - DOI
    1. Lyu S. Deepfake Detection: Current Challenges and Next Steps; Proceedings of the 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW); London, UK. 6–10 July 2020.
    1. Rana S., Sung A.H. DeepfakeStack: A Deep Ensemble-based Learning Technique for Deepfake Detection; Proceedings of the 2020 7th IEEE International Conference on Cyber Security and Cloud Computing (CSCloud)/2020 6th IEEE International Conference on Edge Computing and Scalable Cloud (EdgeCom); New York, NY, USA. 1–3 August 2020; pp. 70–75. - DOI
    1. Tolosana R., Vera-Rodriguez R., Fierrez J., Morales A., Ortega-Garcia J. Deepfakes and beyond: A Survey of face manipulation and fake detection. Inf. Fusion. 2020;64:131–148. doi: 10.1016/j.inffus.2020.06.014. - DOI
    1. Malavida, FakeApp 2.2.0. 2022. [(accessed on 1 March 2022)]. Available online: https://www.malavida.com/en/soft/fakeapp/

LinkOut - more resources