Among other models aimed at understanding self-sustained oscillations in neural networks, the NNLIF model with synaptic delay was developed almost twenty years ago to model fast global oscillations in networks of weakly firing inhibitory neurons. Periodic solutions were numerically observed in this model, but despite its intensive study by researchers in PDEs and probability, there is up-to-date no analytical result on this topic. In this article, we propose to approximate formally these solutions by a Gaussian wave whose periodic movement is described by an associate delay differential equation (DDE). We prove the existence of a periodic solution for this DDE and we give a rigorous asymptotic result on these solutions when the connectivity parameter $b$ goes to $-\infty$. Lastly, we provide heuristic and numerical evidence of the validity of our approximation.