Model-based inexact graph matching on top of DNNs for semantic scene understanding - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Computer Vision and Image Understanding Année : 2023

Model-based inexact graph matching on top of DNNs for semantic scene understanding

Résumé

Deep learning based pipelines for semantic segmentation often ignore structural information available on annotated images used for training. We propose a novel post-processing module enforcing structural knowledge about the objects of interest to improve segmentation results provided by deep learning. This module corresponds to a "many-to-one-or-none" inexact graph matching approach, and is formulated as a quadratic assignment problem. Our approach is compared to a CNN-based segmentation (for various CNN backbones) on two public datasets, one for face segmentation from 2D RGB images (FASSEG), and the other for brain segmentation from 3D MRIs (IBSR). Evaluations are performed using two types of structural information (distances and directional relations, , this choice being a hyper-parameter of our generic framework). On FASSEG data, results show that our module improves accuracy of the CNN by about 6.3% (the Hausdorff distance decreases from 22.11 to 20.71). On IBSR data, the improvement is of 51% (the Hausdorff distance decreases from 11.01 to 5.4). In addition, our approach is shown to be resilient to small training datasets that often limit the performance of deep learning methods: the improvement increases as the size of the training dataset decreases.

Dates et versions

hal-04264704 , version 1 (20-01-2024)

Identifiants

Citer

Jérémy Chopin, Jean-Baptiste Fasquel, Harold Mouchère, Rozenn Dahyot, Isabelle Bloch. Model-based inexact graph matching on top of DNNs for semantic scene understanding. Computer Vision and Image Understanding, 2023, 235, pp.103744. ⟨10.1016/j.cviu.2023.103744⟩. ⟨hal-04264704⟩
30 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More