We characterize the observability property (and, by duality, the controllability and the stabilization) of the wave equation on a Riemannian manifold $\Omega,$ with or without boundary, where the observation (or control) domain is time-varying. We provide a condition ensuring observability, in terms of propagating bicharacteristics. This condition extends the well-known geometric control condition established for fixed observation domains. As one of the consequences, we prove that it is always possible to find a time-dependent observation domain of arbitrarily small measure for which the observability property holds. From a practical point of view, this means that it is possible to reconstruct the solutions of the wave equation with only few sensors (in the Lebesgue measure sense), at the price of moving the sensors in the domain in an adequate way. We provide several illustrating examples, in which the observation domain is the rigid displacement in $\Omega$ of a fixed domain, with speed $v,$ showing that the observability property depends both on $v$ and on the wave speed. Despite the apparent simplicity of some of our examples, the observability property can depend on nontrivial arithmetic considerations.