Memory transformers for full context and high-resolution 3D Medical Segmentation - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Memory transformers for full context and high-resolution 3D Medical Segmentation

Résumé

Transformer models achieve state-of-the-art results for image segmentation. However, achieving long-range attention, necessary to capture global context, with high-resolution 3D images is a fundamental challenge. This paper introduces the Full resolutIoN mEmory (FINE) transformer to overcome this issue. The core idea behind FINE is to learn memory tokens to indirectly model full range interactions while scaling well in both memory and computational costs. FINE introduces memory tokens at two levels: the first one allows full interaction between voxels within local image regions (patches), the second one allows full interactions between all regions of the 3D volume. Combined, they allow full attention over high resolution images, e.g. 512 x 512 x 256 voxels and above. Experiments on the BCV image segmentation dataset shows better performances than state-of-the-art CNN and transformer baselines, highlighting the superiority of our full attention mechanism compared to recent transformer baselines, e.g. CoTr, and nnFormer.
Fichier principal
Vignette du fichier
MLMI___Memory_transformers_for_full_context_and_high_resolution_3D_Medical_Segmentation.pdf (1.07 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03769673 , version 1 (05-09-2022)

Identifiants

Citer

Loic Themyr, Clément Rambour, Nicolas Thome, Toby Collins, Alexandre Hostettler. Memory transformers for full context and high-resolution 3D Medical Segmentation. Machine Learning in Medical Imaging. MLMI 2022, 2022, Singapour, Singapore. ⟨10.48550/arXiv.2210.05313⟩. ⟨hal-03769673⟩
49 Consultations
104 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More