A Generative Model for Texture Synthesis based on Optimal Transport between Feature Distributions - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of Mathematical Imaging and Vision Année : 2022

A Generative Model for Texture Synthesis based on Optimal Transport between Feature Distributions

Résumé

We propose GOTEX, a general framework for texture synthesis by optimization that constrains the statistical distribution of local features. While our model encompasses several existing texture models, we focus on the case where the comparison between feature distributions relies on optimal transport distances. We show that the semi-dual formulation of optimal transport allows to control the distribution of various possible features, even if these features live in a high-dimensional space. We then study the resulting minimax optimization problem, which corresponds to a Wasserstein generative model, for which the inner concave maximization problem can be solved with standard stochastic gradient methods. The alternate optimization algorithm is shown to be versatile in terms of applications, features and architecture; in particular it allows to produce high-quality synthesized textures with different sets of features. We analyze the results obtained by constraining the distribution of patches or the distribution of responses to a pre-learned VGG neural network. We show that the patch representation can retrieve the desired textural aspect in a more precise manner. We also provide a detailed comparison with state-of-the-art texture synthesis methods. The GOTEX model based on patch features is also adapted to texture inpainting and texture interpolation. Finally, we show how to use our framework to learn a feed-forward neural network that can synthesize on-the-fly new textures of arbitrary size in a very fast manner. Experimental results and comparisons with the mainstream methods from the literature illustrate the relevance of the generative models learned with GOTEX.
Fichier principal
Vignette du fichier
2007.03408 (36.19 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03386084 , version 1 (19-12-2023)

Identifiants

Citer

Antoine Houdard, Arthur Leclaire, Nicolas Papadakis, Julien Rabin. A Generative Model for Texture Synthesis based on Optimal Transport between Feature Distributions. Journal of Mathematical Imaging and Vision, 2022, ⟨10.1007/s10851-022-01108-9⟩. ⟨hal-03386084⟩
111 Consultations
1 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More