Combining a Relaxed EM Algorithm with Occam's Razor for Bayesian Variable Selection in High-Dimensional Regression - Archive ouverte HAL Access content directly
Journal Articles Journal of Multivariate Analysis Year : 2015

Combining a Relaxed EM Algorithm with Occam's Razor for Bayesian Variable Selection in High-Dimensional Regression

Abstract

We address the problem of Bayesian variable selection for high-dimensional lin-ear regression. We consider a generative model that uses a spike-and-slab-like prior distribution obtained by multiplying a deterministic binary vector, which traduces the sparsity of the problem, with a random Gaussian parameter vector. The origi-nality of the work is to consider inference through relaxing the model and using a type-II log-likelihood maximization based on an EM algorithm. Model selection is performed afterwards relying on Occam's razor and on a path of models found by the EM algorithm. Numerical comparisons between our method, called spinyReg, and state-of-the-art high-dimensional variable selection algorithms (such as lasso, adap-tive lasso, stability selection or spike-and-slab procedures) are reported. Competitive variable selection results and predictive performances are achieved on both simulated and real benchmark data sets. An original regression data set involving the predic-tion of the number of visitors of the Orsay museum in Paris using bike-sharing system data is also introduced, illustrating the efficiency of the proposed approach. An R package implementing the spinyReg method is currently under development and is available at https://r-forge.r-project.org/projects/spinyreg.
Fichier principal
Vignette du fichier
spinyreg_jmva.pdf (1.98 Mo) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01003395 , version 1 (10-09-2014)
hal-01003395 , version 2 (29-01-2015)

Licence

Identifiers

Cite

Pierre Latouche, Pierre-Alexandre Mattei, Charles Bouveyron, Julien Chiquet. Combining a Relaxed EM Algorithm with Occam's Razor for Bayesian Variable Selection in High-Dimensional Regression. Journal of Multivariate Analysis, 2015, 146, pp.177-190. ⟨10.1016/j.jmva.2015.09.004⟩. ⟨hal-01003395v2⟩
938 View
1018 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More