Diffeomorphism-based feature learning using Poincaré inequalities on augmented input space - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Diffeomorphism-based feature learning using Poincaré inequalities on augmented input space

Résumé

We propose a gradient-enhanced algorithm for high-dimensional function approximation. The algorithm proceeds in two steps: firstly, we reduce the input dimension by learning the relevant input features from gradient evaluations, and secondly, we regress the function output against the pre-learned features. To ensure theoretical guarantees, we construct the feature map as the first components of a diffeomorphism, which we learn by minimizing an error bound obtained using Poincaré Inequality applied either in the input space or in the feature space. This leads to two different strategies, which we compare both theoretically and numerically and relate to existing methods in the literature. In addition, we propose a dimension augmentation trick to increase the approximation power of feature detection. A generalization to vector-valued functions demonstrate that our methodology directly applies to learning autoencoders. Here, we approximate the identity function over a given dataset by a composition of feature map (encoder) with the regression function (decoder). In practice, we construct the diffeomorphism using coupling flows, a particular class of invertible neural networks. Numerical experiments on various high-dimensional functions show that the proposed algorithm outperforms state-of-the-art competitors, especially with small datasets.
Fichier principal
Vignette du fichier
article_preprint.pdf (1.28 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04364208 , version 1 (26-12-2023)

Licence

Paternité

Identifiants

  • HAL Id : hal-04364208 , version 1

Citer

Romain Verdière, Clémentine Prieur, Olivier Zahm. Diffeomorphism-based feature learning using Poincaré inequalities on augmented input space. 2023. ⟨hal-04364208⟩
122 Consultations
108 Téléchargements

Partager

Gmail Facebook X LinkedIn More