Exemplar-based image colorization using object-guided attention - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Exemplar-based image colorization using object-guided attention

Résumé

Exemplar-based image colorization is a challenging task that involves adding color to a grayscale image using a reference color image. The goal is to preserve the semantic content of the target image while also incorporating the color style of the reference image. However, results from previous methods are still unsatisfactory for real-world applications. One of the reasons is that they are inefficient at exploiting semantic color information, mainly when two or more objects are presented in the target or reference images. In this work, we propose a novel end-to-end deep learning framework for exemplar-based colorization that integrates user-provided object masks. We aim to guide the colorization on specific and meaningful objects rather than a full reference image. Our framework consists of an encoder-decoder generator architecture. The core module of the encoder is our proposed masked super-attention. This multiscale object-specific attention mechanism improves the ability to transfer color characteristics from the user's selected objects. In addition, we introduce a strategic method for selecting pertinent target/reference image pairs at the object-level. To comprehensively evaluate the effectiveness of our proposed approach, we conduct a complete evaluation of both full-level and object-level images. Finally, our framework achieves colorful and visually pleasant colorization and surpasses state-of-the-art methods on different quantitative metrics.
Fichier principal
Vignette du fichier
main.pdf (19.2 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04215100 , version 1 (22-09-2023)

Identifiants

  • HAL Id : hal-04215100 , version 1

Citer

Hernan Carrillo, Michaël Clément, Aurélie Bugeau. Exemplar-based image colorization using object-guided attention. 2023. ⟨hal-04215100⟩

Collections

CNRS ANR
47 Consultations
12 Téléchargements

Partager

Gmail Facebook X LinkedIn More