This paper introduces a Bayesian non parametric (BNP) model associated with a Markov random field (MRF) for detecting changes between remote sensing images acquired by homogeneous or heterogeneous sensors. The proposed model is built for an analysis window which takes advantage of the spatial information via an MRF. The model does not require any a priori knowledge about the number of objects contained in the window thanks to the BNP framework. The change detection strategy can be divided into two steps. First, the segmentation of the two images is performed using a region based approach. Second, the joint statistical properties of the objects in the two images allows an appropriate manifold to be defined. This manifold describes the relationships between the different sensor responses to the observed scene and can be learnt from a training unchanged area. It allows us to build a similarity measure between the images that can be used in many applications such as change detection or image registration. Simulation results conducted on synthetic and real optical and synthetic aperture radar (SAR) images show the efficiency of the proposed method for change detection.