On the Importance of Visual Context for Data Augmentation in Scene Understanding - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Pattern Analysis and Machine Intelligence Année : 2019

On the Importance of Visual Context for Data Augmentation in Scene Understanding

Résumé

Performing data augmentation for learning deep neural networks is known to be important for training visual recognition systems. By artificially increasing the number of training examples, it helps reducing overfitting and improves generalization. While simple image transformations can already improve predictive performance in most vision tasks, larger gains can be obtained by leveraging task-specific knowledge. In this work, we consider object detection, semantic and instance segmentation and augment training images by blending objects in existing scenes, using instance segmentation annotations. We observe that randomly pasting objects on images hurts the performance, unless the object is placed in the right context. To resolve this issue, we propose an explicit context model by using a convolutional neural network, which predicts whether an image region is suitable for placing a given object or not. In our experiments, we show that our approach is able to improve object detection, semantic and instance segmentation on the PASCAL VOC12 and COCO datasets, with significant gains in a limited annotation scenario. We also show that the method is not limited to datasets that come with expensive pixel-wise instance annotations and can be used when only bounding boxes are available, by employing weakly-supervised learning for instance masks approximation.
Fichier principal
Vignette du fichier
main.pdf (3.59 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01869784 , version 1 (06-09-2018)
hal-01869784 , version 2 (13-05-2019)
hal-01869784 , version 3 (19-09-2019)
hal-01869784 , version 4 (06-01-2020)

Identifiants

Citer

Nikita Dvornik, Julien Mairal, Cordelia Schmid. On the Importance of Visual Context for Data Augmentation in Scene Understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43 (6), pp.2014-2028. ⟨10.1109/TPAMI.2019.2961896⟩. ⟨hal-01869784v4⟩
464 Consultations
860 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More