Oscillatory systems of interacting Hawkes processes with Erlang memory kernels were introduced in Ditlevsen and Löcherbach (2017). They are piecewise deterministic Markov processes (PDMP) and can be approximated by a stochastic diffusion. First, a strong error bound between the PDMP and the diffusion is proved. Second, moment bounds for the resulting diffusion are derived. Third, approximation schemes for the diffusion , based on the numerical splitting approach, are proposed. These schemes are proved to converge with mean-square order 1 and to preserve the properties of the diffusion, in particular the hypoellipticity, the ergod-icity and the moment bounds. Finally, the PDMP and the diffusion are compared through numerical experiments, where the PDMP is simulated with an adapted thinning procedure.