Bayesian Optimisation is an emerging machine learning technique known to be efficient especially for optimising functions which are expensive to evaluate. In Bayesian Optimisation, a Gaussian Process model of the unknown function is identified based on available data. Its estimate of the unknown function and the associated uncertainties are used to build a so-called acquisition function which does a tradeoff between exploitation and exploration. The latter is then iteratively maximised to find candidates which are promising to be close to the optimum. Despite the effectiveness of this technique, it is interesting to investigate other different models to Gaussian Processes. In this paper an alternative to Bayesian Optimization based on the use of neural networks instead of Gaussian Processes is proposed. Since neural networks do not naturally provide an information about the quality of the estimates, the proposed method makes use of the distance between an evaluated point and the closest observed one to it for the exploration. The efficiency of the proposed approach is illustrated on two different study cases : controller design and aerodynamic design.