In this article, we are interested in the behavior of a fully connected network of $N$ neurons, where $N$ tends to infinity. We assume that neurons follow the stochastic FitzHugh-Nagumo model, whose specificity is the non-linearity with a cubic term. We prove a result of uniform in time propagation of chaos of this model in a mean-field framework. We also exhibit explicit bounds. We use a coupling method initially suggested by A. Eberle (arXiv:1305.1233), and recently extended by (1805.11387), known as the reflection coupling. We simultaneously construct a solution of the $N$ particles system and $N$ independent copies of the non-linear McKean-Vlasov limit such that, by considering an appropriate semimetrics that takes into account the various possible behaviors of the processes, the two solutions tend to get closer together as $N$ increases, uniformly in time. The reflection coupling allows us to deal with the non-convexity of the underlying potential in the dynamics of the quantities defining our network, and show independence at the limit for the system in mean field interaction with sufficiently small Lipschitz continuous interactions.