The Missing Data Encoder: Cross-Channel Image Completion with Hide-And-Seek Adversarial Network - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

The Missing Data Encoder: Cross-Channel Image Completion with Hide-And-Seek Adversarial Network

Arnaud Dapogny
  • Fonction : Auteur
Patrick Pérez
  • Fonction : Auteur
  • PersonId : 1022281

Résumé

Image completion is the problem of generating whole images from fragments only. It encompasses inpainting (generating a patch given its surrounding), reverse inpainting/extrapolation (generating the periphery given the central patch) as well as colorization (generating one or several channels given other ones). In this paper, we employ a deep network to perform image completion, with adversarial training as well as perceptual and completion losses, and call it the ``missing data encoder'' (MDE). We consider several configurations based on how the seed fragments are chosen. We show that training MDE for ``random extrapolation and colorization'' (MDE-REC), i.e. using random channel-independent fragments, allows a better capture of the image semantics and geometry. MDE training makes use of a novel ``hide-and-seek'' adversarial loss, where the discriminator seeks the original non-masked regions, while the generator tries to hide them. We validate our models both qualitatively and quantitatively on several datasets, showing their interest for image completion, unsupervised representation learning as well as face occlusion handling.

Dates et versions

hal-02507454 , version 1 (13-03-2020)

Identifiants

Citer

Arnaud Dapogny, Matthieu Cord, Patrick Pérez. The Missing Data Encoder: Cross-Channel Image Completion with Hide-And-Seek Adversarial Network. AAAI-20 - Thirty-Fourth AAAI Conference on Artificial Intelligence, Feb 2020, New York, New York, United States. ⟨hal-02507454⟩
37 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More