Model Selection for Bayesian Autoencoders - Archive ouverte HAL Access content directly
Preprints, Working Papers, ... Year : 2021

Model Selection for Bayesian Autoencoders

Ba-Hien Tran
  • Function : Author
  • PersonId : 1302369
  • IdRef : 272912042
Pietro Michiardi
  • Function : Author
  • PersonId : 1084771
Edwin V. Bonilla
Maurizio Filippone
  • Function : Author
  • PersonId : 1021042

Abstract

We develop a novel method for carrying out model selection for Bayesian autoencoders (BAEs) by means of prior hyper-parameter optimization. Inspired by the common practice of type-II maximum likelihood optimization and its equivalence to Kullback-Leibler divergence minimization, we propose to optimize the distributional sliced-Wasserstein distance (DSWD) between the output of the autoencoder and the empirical data distribution. The advantages of this formulation are that we can estimate the DSWD based on samples and handle high-dimensional problems. We carry out posterior estimation of the BAE parameters via stochastic gradient Hamiltonian Monte Carlo and turn our BAE into a generative model by fitting a flexible Dirichlet mixture model in the latent space. Consequently, we obtain a powerful alternative to variational autoencoders, which are the preferred choice in modern applications of autoencoders for representation learning with uncertainty. We evaluate our approach qualitatively and quantitatively using a vast experimental campaign on a number of unsupervised learning tasks and show that, in small-data regimes where priors matter, our approach provides state-of-the-art results, outperforming multiple competitive baselines.

Dates and versions

hal-03344782 , version 1 (15-09-2021)

Identifiers

Cite

Ba-Hien Tran, Simone Rossi, Dimitrios Milios, Pietro Michiardi, Edwin V. Bonilla, et al.. Model Selection for Bayesian Autoencoders. 2021. ⟨hal-03344782⟩
39 View
0 Download

Altmetric

Share

Gmail Facebook X LinkedIn More