Future-State Predicting LSTM for Early Surgery Type Recognition - Ecole Nationale du Génie de l'Eau et de l'Environnement de Strasbourg Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Medical Imaging Année : 2020

Future-State Predicting LSTM for Early Surgery Type Recognition

Siddharth Kannan
  • Fonction : Auteur
Gaurav Yengera
  • Fonction : Auteur
Didier Mutter
  • Fonction : Auteur
Jacques Marescaux
  • Fonction : Auteur

Résumé

This work presents a novel approach for the early recognition of the type of a laparoscopic surgery from its video. Early recognition algorithms can be beneficial to the development of "smart" OR systems that can provide automatic context-aware assistance, and also enable quick database indexing. The task is however ridden with challenges specific to videos belonging to the domain of laparoscopy, such as high visual similarity across surgeries and large variations in video durations. To capture the spatio-temporal dependencies in these videos, we choose as our model a combination of a convolutional neural network (CNN) and long short-term memory (LSTM) network. We then propose two complementary approaches for improving early recognition performance. The first approach is a CNN fine-tuning method that encourages surgeries to be distinguished based on the initial frames of laparoscopic videos. The second approach, referred to as " Future-State Predicting LSTM," trains an LSTM to predict information related to future frames, which helps in distinguishing between the different types of surgeries. We evaluate our approaches on a large dataset of 425 laparoscopic videos containing nine types of surgeries (Laparo425), and achieve on average an accuracy of 75% having observed only the first 10 min of a surgery. These results are quite promising from a practical standpoint and also encouraging for other types of image-guided surgeries.

Dates et versions

hal-03245668 , version 1 (01-06-2021)

Identifiants

Citer

Siddharth Kannan, Gaurav Yengera, Didier Mutter, Jacques Marescaux, Nicolas Padoy. Future-State Predicting LSTM for Early Surgery Type Recognition. IEEE Transactions on Medical Imaging, 2020, 39 (3), pp.556-566. ⟨10.1109/TMI.2019.2931158⟩. ⟨hal-03245668⟩
21 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More