Talk on: "Combination of Gaussian Process Models for High-Dimensional Bayesian Optimization"
Résumé
Gaussian Processes (GPs) are commonly used as surrogate models of expensive computer codes in various applications. In particular, in engineering design optimization, Bayesian approaches based on GPs such as efficient global optimization (EGO) are employed to speed-up the optimization process by reducing the number of computer code evaluations. These methods have been successfully applied to many real-world applications in low dimensions (less than 30).
However, engineering designs are often parametrized by more than 50 shape parameters in practice. In higher dimensions, GPs suffer from the curse of dimensionality and building an accurate surrogate model is met with various setbacks. One of the main challenges is related to optimizing the covariance length-scale hyperparameters of the GP. Anisotropic GP models consider one length-scale hyperparameter per dimension whose optimization is typically performed by maximizing the log-likelihood of the model. In high-dimension, this optimization is problematic due to the exponential growth of the search space with the dimension, to the computational cost of the log-likelihood, and to over-fitting issues when there are too few observations. For these reasons, maximum likelihood estimation of the hyperparameters often fails to provide correct values.
In this talk, we present a new method for high-dimensional GP models which bypasses the length-scales optimization by combining sub-models with random length-scales. We obtain a closed-form expression of the combination that does not require any inner optimization. We also describe an approach to sample suitable length-scales for the sub-models using a criterion based on the entropy of the correlations. Finally, we present a way to obtain the prediction variance of the combination for any weighting methods. We apply our combined model to high-dimensional EGO for the design of an electric machine. We show that the classical GP approach using maximum likelihood estimation fails to properly optimize the length-scale hyperparameters and that our method successfully builds more accurate surrogate models for EGO, thus reducing the number of computer code evaluations to obtain optimal designs.