The ensemble Kalman filter is widely used in applications because, for high dimensional filtering problems, it has a robustness that is not shared for example by the particle filter; in particular it does not suffer from weight collapse. However, there is no theory which quantifies its accuracy as an approximation of the true filtering distribution, except in the Gaussian setting. To address this issue we provide the first analysis of the accuracy of the ensemble Kalman filter beyond the Gaussian setting. Our analysis is developed for the mean field ensemble Kalman filter. We rewrite this filter in terms of maps on probability measures, and then we prove that these maps are locally Lipschitz in an appropriate weighted total variation metric. Using these stability estimates we demonstrate that, if the true filtering distribution is close to Gaussian after appropriate lifting to the joint space of state and data, then it is well approximated by the ensemble Kalman filter. Finally, we provide a generalization of these results to the Gaussian projected filter, which can be viewed as a mean field description of the unscented Kalman filter.