We study a purely inhibitory neural network model where neurons are represented by their state of inhibition. The study we present here is partially based on the work of Cottrell [3] and Fricker et al. [5]. The spiking rate of a neuron depends only on its state of inhibition. When a neuron spikes, its state is replaced by a random new state, independently of anything else and the inhibition state of the other neurons increase by a positive value. Using the Perron-Frobenius theorem, we show the existence of a Lyapunov function for the process. Furthermore, we prove a local Doeblin condition which implies the existence of an invariant measure for the process. Finally, we extend our model to the case where the neurons are indexed by Z. We construct a perfect simulation algorithm to show the recurrence of the process under certain conditions. To do this, we rely on the classical contour technique used in the study of contact processes, and assuming that the spiking rate lies on the interval [β_ * , β^ * ], we show that there is a critical threshold for the ratio δ = β _* /(β^ * −β_ *) over which the process is ergodic.