The Nearest Neighbor entropy estimate: an adequate tool for adaptive MCMC evaluation - Archive ouverte HAL
Rapport Année : 2014

The Nearest Neighbor entropy estimate: an adequate tool for adaptive MCMC evaluation

Résumé

Many recent and often adaptive Markov Chain Monte Carlo (MCMC) methods are associated in practice to unknown rates of convergence. We propose a simulation-based methodology to estimate MCMC efficiency, grounded on a Kullback divergence criterion requiring an estimate of the entropy of the algorithm successive densities, computed from iid simulated chains. We recently proved in Chauveau and Vandekerkhove (2013) some consistency results in MCMC setup for an entropy estimate based on Monte-Carlo integration of a kernel density estimate based on Györfi and Van Der Meulen (1989). Since this estimate requires some tuning parameters and deteriorates as dimension increases, we investigate here an alternative estimation technique based on Nearest Neighbor estimates. This approach has been initiated by Kozachenko and Leonenko (1987) but used mostly in univariate situations until recently when entropy estimation has been considered in other fields like neuroscience. Theoretically, we prove that, under certain uniform control conditions, the successive densities of a generic class of Adaptive Metropolis-Hastings algorithms to which most of the strategies proposed in the recent literature belong can be estimated consistently with our method. We then show that in MCMC setup where moderate to large dimensions are common, this estimate seems appealing for both computational and operational considerations, and that the problem inherent to a non neglictible bias arising in high dimension can be overcome. All our algorithms for MCMC simulation and entropy estimation are implemented in an R package taking advantage of recent advances in high performance (parallel) computing.
Fichier principal
Vignette du fichier
Chauveau_NN_AMCMC.pdf (1.74 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01068081 , version 1 (24-09-2014)

Identifiants

  • HAL Id : hal-01068081 , version 1

Citer

Didier Chauveau, Pierre Vandekerkhove. The Nearest Neighbor entropy estimate: an adequate tool for adaptive MCMC evaluation. 2014. ⟨hal-01068081⟩
427 Consultations
570 Téléchargements

Partager

More