Demystifying Attention Mechanisms for Deepfake Detection - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Demystifying Attention Mechanisms for Deepfake Detection

Résumé

Manipulated images and videos, i.e., deepfakes have become increasingly realistic due to the tremendous progress of deep learning methods. However, such manipulation has triggered social concerns, necessitating the introduction of robust and reliable methods for deepfake detection. In this work, we explore a set of attention mechanisms and adapt them for the task of deepfake detection. Generally, attention mechanisms in videos modulate the representation learned by a convolutional neural network (CNN) by focusing on the salient regions across space-time. In our scenario, we aim at learning discriminative features to take into account the temporal evolution of faces to spot manipulations. To this end, we address the two research questions ‘How to use attention mechanisms?’, and ‘What type of attention is effective for the task of deepfake detection?’ Towards answering these questions, we provide a detailed study and experiments on videos tampered by four manipulation techniques, as included in the FaceForensics++ dataset. We investigate three scenarios, where the networks are trained to detect (a) all manipulated videos, (b) each manipulation technique individually, as well as (c) the veracity of videos pertaining to manipulation techniques not included in the train set.
Fichier principal
Vignette du fichier
FG21_deepfake_Das_Das_Dantcheva.pdf (1.27 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03536498 , version 1 (19-01-2022)

Identifiants

Citer

Abhijit Das, Srijan Das, Antitza Dantcheva. Demystifying Attention Mechanisms for Deepfake Detection. FG 2021 - IEEE International Conference on Automatic Face and Gesture Recognition, Dec 2021, virtual, India. ⟨10.1109/FG52635.2021.9667026⟩. ⟨hal-03536498⟩
47 Consultations
280 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More