Article Dans Une Revue Computers and Electrical Engineering Année : 2025

Integrating perceptual quality analysis and caption-based features for robust deepfake video detection

Résumé

Inspired by the benefits of both linguistic representations as well as visual quality analysis, we propose a multimodal framework for deepfake detection with reliable accuracy. Our method acquires low-level perceptual features from frames of video, including contrast, brightness, and sharpness, and computes artifact scores for artificial anomaly detection. In parallel, our method produces descriptive frame-level captions, summed up for generating video-level summaries for capturing contextual coherence. For training the model, our method leverages the FaceForensics++ dataset, comprising several techniques of deepFake manipulation, including DeepFake, Face2Face, FaceSwap, as well as NeuralTextures. Metadata comprised of quality measurements, the artifact scores, as well as textual captions, is tokenized, processed by a DeepSeek V2 Lite model, fine-tuned by the Low-Rank Adaptation (LoRA) procedure, as the backbone for classification. We test the proposed model on both intra-dataset as well as cross-dataset settings, demonstrating good accuracy as well as impressive generalization ability. Our method achieves a 96.51% accuracy for classifying, by integrating low-level perceptual as well as high-level semantic reason, effectively marking manipulated content as well as upholding the integrity of the medium in a world of increasingly evolving generative methods. The employed source code as well as dataset preprocessing scripts for our work are made available for public access on: https://github.com/HessenUPHF/perceptual-caption-detection

Fichier principal
Vignette du fichier
Hessen Bougueffa Eutamene_1-s2.0-S0045790625006421-main.pdf (3.27 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05254097 , version 1 (10-10-2025)

Licence

Identifiants

Citer

Hessen Bougueffa Eutamene, Wassim Hamidouche, Mamadou Keita, Abdelmalik Taleb-Ahmed, Abdenour Hadid. Integrating perceptual quality analysis and caption-based features for robust deepfake video detection. Computers and Electrical Engineering, 2025, 128 (Part A), pp.110699. ⟨10.1016/j.compeleceng.2025.110699⟩. ⟨hal-05254097⟩
1410 Consultations
343 Téléchargements

Altmetric

Partager

  • More