To be deployed in safety critical applications, neural network (NN) systems require to be verified during the development phase and then monitored during the runtime phase. The latter phase is essential to closely supervise the performance and the behavior of NNs, particularly when used for safety-related tasks. This paper presents a novel approach for monitoring NN classifiers through real-time supervision of the model's behavior and decisions, to detect potential anomalies. The approach is based on the concept of Neuron Activation Paths (NAPath), which allows extracting relevant activated/inactivated paths that link the inputs to the outputs of the network and significantly influence the NN's classification decision. The main idea is to characterize paths for each class using training data, i.e., from the training data set, we group the images of the same class together. Then, for each group of images, we identify their common active and inactive paths, respectively. The sets of active and inactive paths constitute a NAPath, which is used as a signature to feature the corresponding class. The monitoring system then uses these NAPaths online to continuously check whether the paths activated by the image fit the NAPath characterization associated with its classification, as returned by the network. The monitoring system raises alarms if an abnormal decision of the network is detected. We evaluated our approach on a benchmark of neural networks pre-trained on the MNIST data set.