Deep Neural Networks (DNNs) are increasingly used in safety critical autonomous systems. In this paper, we present MOZART, a DNN accelerator architecture which provides fault detection and fault tolerance. MOZART is a systolic architecture based on the Output Stationary (OS) variant, as it is the one that inherently limits fault propagation. In addition, MOZART achieves fault detection with on-line functional testing of the Processing Elements (PEs). Faulty PEs are swiftly taken off-line with minimal classification impact. The implementation of our approach on Squeezenet results in a loss of accuracy of less than 3% in the presence of a single faulty PE, compared to 15-33% without mitigation. The area overhead for the test logic does not exceed 8%. Dropout during training further improves fault tolerance, without a priori knowledge of the faults.