Hyperparameter optimization of deep neural networks: combining Hperband with Bayesian model selection - Archive ouverte HAL Access content directly
Conference Papers Year : 2017

Hyperparameter optimization of deep neural networks: combining Hperband with Bayesian model selection

(1, 2) , (3) , , (1, 2)
1
2
3

Abstract

One common problem in building deep learning architectures is the choice of the hyper-parameters. Among the various existing strategies, we propose to combine two complementary ones. On the one hand, the Hyperband method formalizes hyper-parameter optimization as a resource allocation problem, where the resource is the time to be distributed between many configurations to test. On the other hand, Bayesian optimization tries to model the hyper-parameter space as efficiently as possible to select the next model to train. Our approach is to model the space with a Gaussian process and sample the next group of models to evaluate with Hyperband. Preliminary results show a slight improvement over each method individually, suggesting the need and interest for further experiments.
Not file

Dates and versions

hal-02412262 , version 1 (15-12-2019)

Identifiers

  • HAL Id : hal-02412262 , version 1

Cite

Hadrien Bertrand, Roberto Ardon, Matthieu Perrot, Isabelle Bloch. Hyperparameter optimization of deep neural networks: combining Hperband with Bayesian model selection. CAp, 2017, Grenoble, France. ⟨hal-02412262⟩
84 View
0 Download

Share

Gmail Facebook Twitter LinkedIn More