Deep Neural Networks are achieving great success in various fields. However, their use remains limited to non critical applications because their behavior is unpredictable and unsafe. In this paper we propose some fault tolerant approaches based on diversifying learning in order to improve DNNs dependability and particularly safety. Our main goal is to increase trust in the outcome of deep learning mechanisms by recognizing the unlearned inputs and preventing misclassification.