This paper presents a new algorithm to detect moving objects within a scene acquired by a stationary camera. A simple recursive non linear operator, the Sigma-Delta filter, is used to estimate two orders of temporal statistics for every pixel of the image. The output data provide a scene characterization allowing a simple and efficient pixel-level change detection framework. For a more suitable detection, exploiting spatial correlation in these data is necessary. We use them as a multiple observation field in a Markov model, leading to a spatiotemporal regularization of the pixel-level solution. This method yields a good trade-off in terms of robustness and accuracy, with a minimal cost in memory and a low computational complexity.