In this article, we propose a method exploiting irregular and unaligned Sentinel-2 satellite image time series (SITS) for large-scale land cover pixel-based classification. We perform end-to-end learning by combining a time and space informed kernel interpolator with a Sparse Variational Gaussian Processes (SVGP) classifier. The interpolator embeds irregular and unaligned SITS onto a fixed and reduced size latent representation. The spatial information is taken into account by using a spatial positional encoding. The obtained latent representation is given to the SVGP classifier and all the parameters are jointly optimized w.r.t. the classification task. We run experiments with irregular and unaligned Sentinel-2 SITS of the full year 2018 over an area of 200 000 km2 (about two billion pixels) in the south of France (27 MGRS tiles). Such experimental condition exacerbates the irregular and unaligned issues of SITS. In terms of overall accuracy, with the learned latent representation instead of linearly interpolated SITS, the results of the SVGP classifier are improved by about 10 points. Moreover, with the learned latent representation, the SVGP classifier outperforms the main state-of-the-art methods from the literature at large scale (e.g., seven points for the Multi-layer Perceptron) and is robust to the available timestamps used for training and testing.