Context. The magnetic field in spiral galaxies is known to have a large-scale spiral structure along the galactic disk and is observed as X-shaped in the halo of some galaxies. While the disk field can be well explained by dynamo action, the three-dimensional structure of the halo field and its physical nature are still unclear. Aims: As first steps towards understanding the halo fields, we want to clarify whether or not the observed X-shaped field is a wide-spread pattern in the halos of spiral galaxies. We also aim to investigate whether these halo fields are simply turbulent fields ordered by compression or shear (anisotropic turbulent fields), or have a large-scale regular structure. Methods: Analysis of the Faraday rotation in the halo is used as a tool to distinguish anisotropic turbulent fields from large-scale magnetic fields. However, this has been challenging until recently because of the faint halo emission in linear polarization. Our sensitive VLA broadband observations in C-band and L-band of 35 spiral galaxies seen edge-on (called CHANG-ES) allowed us to perform rotation measure synthesis (RM synthesis) in their halos and to analyze the results. We further accomplished a stacking of the observed polarization maps of 28 CHANG-ES galaxies in C-band. Results: Though the stacked edge-on galaxies were of different Hubble type, and had differing star formation activity and interaction activity, the stacked image clearly reveals an X-shaped structure of the apparent magnetic field. We detected a large-scale (coherent) halo field in all 16 galaxies that have extended polarized emission in their halos. We detected large-scale field reversals in all of their halos. In six galaxies, these are along lines that are approximately perpendicular to the galactic midplane (vertical RMTL) with about 2 kpc separation. Only in NGC 3044 and possibly in NGC 3448 did we observe vertical giant magnetic ropes (GMR) similar to those detected recently in NGC 4631. Conclusions: The observed X-shaped structure of the halo field seems to be an underlying feature of spiral galaxies. It can be regarded as the two-dimensional projection of the regular magnetic field which we found to have scales of typically 1 kpc or larger observed over several kiloparsecs. The ordered magnetic field extends far out in the halo and beyond. We detected large-scale magnetic field reversals in the halo that may indicate that GMR are more or less tightly wound. With these discoveries, we hope to stimulate model simulations for the halo magnetic field that should also explain the determined asymmetry of the polarized intensity (PI).