A single-pixel camera is a spatial-multiplexing device that reconstructs an image from a sequence of projections of the scene onto some patterns. This architecture is used, for example, to assist neurosurgery with hyperspectral imaging. However, capturing dynamic scenes is very challenging: as the different projections measure different frames of the scene, standard reconstruction approaches suffer from strong motion artifacts. This paper presents a general framework to reconstruct a moving scene with two main contributions. First, we extend the field of view of the camera beyond that defined by the spatial light modulator, which dramatically reduces the model mismatch. Second, we propose to build the dynamic system matrix without warping the patterns, effectively dismissing discretization errors. Numerical experiments show that both our contributions are necessary for an artifact-free reconstruction. The influence of a reduced measured set, robustness to noise and to motion errors were also evaluated.