On the use of GNN-based structural information to improve CNN-based semantic image segmentation - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of Visual Communication and Image Representation Année : 2024

On the use of GNN-based structural information to improve CNN-based semantic image segmentation

Résumé

Convolutional neural networks (CNNs) are widely used for semantic image segmentation across various fields (medicine, robotics), capturing local pixel dependencies for good results. Nevertheless, CNNs struggle to grasp global contextual representations, sometimes leading to structural inconsistencies. Recent approaches aim to broaden their scope using attention mechanisms or deep models, resulting in heavy-weight architectures. To boost CNN performance in semantic segmentation, we propose using a graph neural network (GNN) as a post-processing step. The GNN conducts node classification on appropriately coarsened graphs encoding class probabilities and structural information related to regions segmented by the CNN. The proposal, applicable to any CNN producing a segmentation map, is evaluated on several CNN architectures, using two public datasets (FASSEG and IBSR), with four graph convolution operators. Results reveal performance improvements, enhancing on average the Hausdorff distance by 24.3% on FASSEG and by 74.0% on IBSR. Furthermore, our approach demonstrates resilience to small training datasets.
Fichier principal
Vignette du fichier
elsarticle_YJVCI_template.pdf (3 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04576045 , version 1 (17-05-2024)

Licence

Identifiants

Citer

Patty Coupeau, Jean-Baptiste Fasquel, Mickael Dinomais. On the use of GNN-based structural information to improve CNN-based semantic image segmentation. Journal of Visual Communication and Image Representation, 2024, 101, pp.104167. ⟨10.1016/j.jvcir.2024.104167⟩. ⟨hal-04576045⟩
0 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More