HCiT: Deepfake Video Detection Using a Hybrid Model of CNN features and Vision Transformer - Archive ouverte HAL
Communication Dans Un Congrès Année : 2021

HCiT: Deepfake Video Detection Using a Hybrid Model of CNN features and Vision Transformer

Résumé

The number of new falsified video contents is dramatically increasing, making the need to develop effective deepfake detection methods more urgent than ever. Even though many existing deepfake detection approaches show promising results, the majority of them still suffer from a number of critical limitations. In general, poor generalization results have been obtained under unseen or new deepfake generation methods. Consequently, in this paper, we propose a deepfake detection method called HCiT, which combines Convolutional Neural Network (CNN) with Vision Transformer (ViT). The HCiT hybrid architecture exploits the advantages of CNN to extract local information with the ViT's self-attention mechanism to improve the detection accuracy. In this hybrid architecture, the feature maps extracted from the CNN are feed into ViT model that determines whether a specific video is fake or real. Experiments were performed on Faceforensics++ and DeepFake Detection Challenge preview datasets, and the results show that the proposed method significantly outperforms the state-of-the-art methods. In addition, the HCiT method shows a great capacity for generalization on datasets covering various techniques of deepfake generation. The source code is available at: https://github.com/KADDAR-Bachir/HCiT
Fichier non déposé

Dates et versions

hal-03540151 , version 1 (23-01-2022)

Identifiants

Citer

Bachir Kaddar, Sid Ahmed Fezza, Wassim Hamidouche, Zahid Akhtar, Abdenour Hadid. HCiT: Deepfake Video Detection Using a Hybrid Model of CNN features and Vision Transformer. 2021 International Conference on Visual Communications and Image Processing (VCIP), Dec 2021, Munich, Germany. pp.1-5, ⟨10.1109/VCIP53242.2021.9675402⟩. ⟨hal-03540151⟩
131 Consultations
0 Téléchargements

Altmetric

Partager

More