A new finite element heterogeneous multiscale method (FE-HMM) is proposed for the numerical solution of the wave equation over long times in a rapidly varying medium. Our FE-HMM captures long-time dispersive effects of the true solution at a cost similar to that of a standard numerical homogenization scheme which, however, only captures the short-time macroscale behavior of the wave field.