This paper explores different aspects of semantic segmentation of remote sensing data using deep neural networks. Learning with deep neural networks was revolutionized by the creation of ImageNet, a massive, annotated image dataset. Remote sensing benefited of these new techniques, however Earth Observation datasets remain small in comparison. In this work, we investigate how we can progress towards the ImageNet of remote sensing. In particular, two questions are addressed in this paper. First, how robust are existing supervised learning strategies with respect to data? Second, is it possible-and with what benefits?-to create a dataset equivalent to ImageNet for remote sensing? The main contributions of this work are: (i) a strong robustness analysis of existing supervised learning strategies with respect to remote sensing data, (ii) the introduction of a new dataset-the MiniFrance dataset-as a first step towards the ImageNet of remote sensing.