Bayesian Optimization is a powerful machine learning tool enabling automated design of fixed-structure controllers. A sequence of closed-loop calibration experiments is performed, and the next configuration to be tested is selected by the optimization algorithm in order to minimize an objective function measured directly on the real system. While the approach has been shown to be effective, its applicability is limited in certain domains by safety considerations, as the algorithm may suggest controller configurations which lead to dangerous behaviours in some of the calibration experiments. In this paper, we modify the standard Bayesian Optimization algorithm by introducing explicit constraints for safe exploration of the controller configuration space. The constraints are derived based on a preliminary model of the process dynamics, which is assumed to be available. Aspects for efficient implementation of the proposed methodology are discussed. Simulation examples highlight the advantage of the proposed methodology for controller calibration over the plain Bayesian Optimization algorithm.