A study of attention information from transformer layers in hybrid medical image segmentation networks - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

A study of attention information from transformer layers in hybrid medical image segmentation networks

Résumé

Transformer models have recently started gaining popularity in Computer Vision related tasks. Within Medical Image Segmentation, segmentation models such as TransUNet have incorporated transformer blocks alongside convolutional blocks while remaining faithful to the encoder-decoder based architecture popularized by U-Net. The rationale behind this is to supplement the local information obtained from convolutional kernels with the global information obtained from transformer blocks. The present work examines information flow with a focus on attention values within transformer blocks of three such segmentation models: (i) TransUNet, (ii) 2D CATS (Complementary CNN and Transformer Encoders for Segmentation), and (iii) 2D UNETR (UNEt TRansformers). For each of these models, an analysis of attention information reveals as to how many transformer blocks are necessary in order to effectively achieve a global receptive field. Based on this, compressed versions of these models are proposed, helping reduce the number of model parameters to around 40% of the original parameters for 2D CATS and around 25% for TransUNet and 2D UNETR. With the help of three different datasets (IBSR 18, EMIDEC, Synapse multi-organ), it is shown that in terms of the dice metric the performance of the compressed model does not drop by more than 5% compared to the original model.
Fichier principal
Vignette du fichier
SPIE_Proceedings__Style_template_and_guidelines_for_authors__1_ (1).pdf (3.09 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03965497 , version 1 (31-01-2023)

Identifiants

Citer

Syed Nouman Hasany, Caroline Petitjean, Fabrice Meriaudeau. A study of attention information from transformer layers in hybrid medical image segmentation networks. SPIE Medical Imaging, Feb 2023, San Diego, United States. ⟨10.1117/12.2652215⟩. ⟨hal-03965497⟩
50 Consultations
107 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More