Semantic-driven Colorization - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Semantic-driven Colorization

M.M. Ho
  • Fonction : Auteur
J. Zhou
  • Fonction : Auteur

Résumé

Recent colorization works implicitly predict the semantic information while learning to colorize black-and-white images. Consequently, the generated color is easier to be overflowed, and the semantic faults are invisible. According to human experience in colorization, our brains first detect and recognize the objects in the photo, then imagine their plausible colors based on many similar objects we have seen in real life, and finally colorize them, as described in Figure 1. In this study, we simulate that human-like action to let our network first learn to understand the photo, then colorize it. Thus, our work can provide plausible colors at a semantic level. Plus, the semantic information predicted from a well-trained model becomes understandable and able to be modified. Additionally, we also prove that Instance Normalization is also a missing ingredient for image colorization, then re-design the inference flow of U-Net to have two streams of data, providing an appropriate way of normalizing the features extracted from the black-and-white image. As a result, our network can provide plausible colors competitive to the typical colorization works for specific objects. Our interactive application is available at https://github.com/minhmanho/semantic-driven_colorization. © 2021 ACM.

Dates et versions

hal-03468792 , version 1 (07-12-2021)

Identifiants

Citer

M.M. Ho, Lu Zhang, A. Raake, J. Zhou. Semantic-driven Colorization. 18th ACM SIGGRAPH European Conference on Visual Media Production, CVMP 2021, Dec 2021, London, United Kingdom. ⟨10.1145/3485441.3485645⟩. ⟨hal-03468792⟩
25 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More