End-to-End Multi-Slice-to-Volume Concurrent Registration and Multimodal Generation - Archive ouverte HAL
Communication Dans Un Congrès Année : 2022

End-to-End Multi-Slice-to-Volume Concurrent Registration and Multimodal Generation

Résumé

For interventional procedures, a real-time mapping between treatment guidance images and planning data is challenging yet essential for successful therapy implementation. Because of time and machine constraints, it involves imaging of different modalities, resolutions and dimensions, along with severe out-of-plane deformations to handle. In this paper, we introduce MSV-RegSyn-Net, a novel, scalable, deep learning-based framework for concurrent slice-to-volume registration and high-resolution modality transfer synthesis. It consists of an end-to-end pipeline made up of (i) a cycle generative adversarial network for multimodal image translation combined with (ii) a multi-slice-to-volume deformable registration network. The concurrent nature of our approach creates mutual benefit for both tasks: image translation is naturally eased by explicit handling of out-of-plane deformations while registration benefits from bringing multimodal signals into the same domain. Our model is fully unsupervised and does not require any ground-truth deformation or segmentation mask. It obtains superior qualitative and quantitative performance for multi-slice MR to 3D CT pelvic imaging compared to state-of-the-art traditional and learning-based methods on both tasks.
Fichier non déposé

Dates et versions

hal-04514722 , version 1 (21-03-2024)

Licence

Identifiants

Citer

Amaury Leroy, Marvin Lerousseau, Théophraste Henry, Alexandre Cafaro, Nikos Paragios, et al.. End-to-End Multi-Slice-to-Volume Concurrent Registration and Multimodal Generation. The 25th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2022), The MICCAI Society, Sep 2022, Singapour, Singapore. pp.152-162, ⟨10.1007/978-3-031-16446-0_15⟩. ⟨hal-04514722⟩
8 Consultations
0 Téléchargements

Altmetric

Partager

More