A hybrid transformer with domain adaptation using interpretability techniques for the application to the detection of risk situations - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Multimedia Tools and Applications Année : 2024

A hybrid transformer with domain adaptation using interpretability techniques for the application to the detection of risk situations

Résumé

Multimedia approaches are strongly required in multi-modal data processing for the detection and recognition of specific events in the data. Hybrid architectures with time series and image/video inputs in the framework of twin CNNs have shown increased performances compared to mono-modal approaches. Pre-trained models have been used in transfer learning to fine-tune the last few layers in the network. This often leads to distribution shifts in the domain. In a real-world scenario, the distribution shifts between the source and target domains can yield poor classification results. With interpretable techniques used in deep neural networks, important features can be highlighted not only for trained models but also reinforced in the training process. Hence the initialization of the target domain model can be performed with improved weights. During data transfer between datasets, the dimensions of the data are also different. We propose a method for model transfer with the adaptation of data dimension and improved initialization with interpretability approaches.
Fichier non déposé

Dates et versions

hal-04538657 , version 1 (09-04-2024)

Identifiants

Citer

Rupayan Mallick, Jenny Benois-Pineau, Akka Zemmari, Kamel Guerda, Boris Mansencal, et al.. A hybrid transformer with domain adaptation using interpretability techniques for the application to the detection of risk situations. Multimedia Tools and Applications, 2024, ⟨10.1007/s11042-024-18687-x⟩. ⟨hal-04538657⟩

Collections

CNRS U1219
0 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More