Bayesian optimisation is an emerging machine learning technique known to be efficient especially for optimising functions which are expensive to evaluate. In Bayesian optimisation, a Gaussian process model of the unknown function is identified based on available data. Its estimate of the unknown function and the associated uncertainties are used to build a so-called acquisition function which does a tradeoff between exploitation and exploration. The latter is then iteratively maximised to find candidates which are promising to be close to the optimum. In this paper, an alternative version of Bayesian optimisation, where the Gaussian process model is replaced by a neural network model, is proposed. As shown in the numerical illustration of this paper, this alternative version will require less computation time when facing optimisation problems with initially large data sets. Since neural networks do not naturally provide an information about the quality of the estimates, a different strategy for the exploration objective of our approach is proposed. The efficiency of the proposed approach is illustrated and compared to Bayesian optimisation on different case studies.