The standard way to train neural-network-based solutions in healthcare does not consider clinical criteria, leading to models that are not necessarily clinically acceptable. In this study, we look at this problem from the perspective of the forecasting of future glucose values of people with diabetes. We propose a new training methodology that achieves the best possible tradeoff between accuracy and medical requirements set by health authorities. Starting from a solution maximizing the prediction accuracy, we progressively relax the accuracy constraints to focus more on the medical ones. This is achieved by considering a new loss function specifically designed for glucose prediction. We evaluate the proposed approach on both people with type-1 and type-2 diabetes. We show that it improves the clinical acceptability of the predictions. Moreover, for given clinical criteria, we are able to find the optimal solution that maximizes the accuracy while at the same time meeting clinical the criteria.