Elastic network models (ENMs) are valuable tools for investigating collective motions of proteins, and a rich variety of simple models have been proposed over the past decade. A good representation of the collective motions requires a good approximation of the covariances between the fluctuations of the individual atoms. Nevertheless, most studies have validated such models only by the magnitudes of the single-atom fluctuations they predict. In the present study, we have quantified the agreement between the covariance structure predicted by molecular dynamics (MD) simulations and those predicted by a representative selection of proposed coarse-grained ENMs. We then contrast this approach with the comparison to MD-predicted atomic fluctuations and comparison to crystallographic B-factors. While all the ENMs yield approximations to the MD-predicted covariance structure, we report large and consistent differences between proposed models. We also find that the ability of the ENMs to predict atomic fluctuations is correlated with their ability to capture the covariance structure. In contrast, we find that the models that agree best with B-factors model collective motions less reliably and recommend against using B-factors as a benchmark.