We report the first experimental test of the topological phase predicted by He and McKellar and by Wilkens in 1993: this phase, which appears when an electric dipole propagates in a magnetic field, is connected to the Aharonov-Casher effect by electric-magnetic duality. The He-McKellar-Wilkens phase is quite small, at most $27$ mrad in our experiment, and this experiment requires the high phase sensitivity of our atom interferometer with spatially separated arms as well as symmetry reversals such as the direction of the electric and magnetic fields. The measured value of the He-McKellar-Wilkens phase differs by $31$\% from its theoretical value, a difference possibly due to some as yet uncontrolled systematic errors.