Article Dans Une Revue IEEE MultiMedia Année : 2022

Detection of Risky Situations for Frail Adults With Hybrid Neural Networks on Multimodal Health Data

Résumé

In healthcare applications, the multimedia methodology is applied to multimodal signals and visual data. This article focuses on the detection of risk situations of frail people from lifelog multimodal signals and video recorded with wearable sensors. We propose a hybrid 3D convolutional neural network (3DCNN) and gated recurrent unit (GRU) (3DCNN-GRU) deep architecture with two branches. The first branch is a GRU network with a global attention block for classification of multisensory signal data. The second branch is a 3DCNN with windowing synchronized with multidimensional time-series signals. Two branches of the neural network are fused yielding promising results. The method produces 83.26% accuracy with dataset BIRDS. Benchmarking is also fulfilled on a publicly available dataset in action recognition.

Fichier non déposé

Dates et versions

hal-03723815 , version 1 (15-07-2022)

Identifiants

Citer

Rupayan Mallick, Thinhinane Yebda, Jenny Benois Pineau, Akka Zemmari, Marion Pech, et al.. Detection of Risky Situations for Frail Adults With Hybrid Neural Networks on Multimodal Health Data. IEEE MultiMedia, 2022, 29 (1), pp.7-17. ⟨10.1109/Mmul.2022.3147381⟩. ⟨hal-03723815⟩
49 Consultations
0 Téléchargements

Altmetric

Partager

  • More