Paving the way toward foundation models for irregular and unaligned Satellite Image Time Series - Archive ouverte HAL
Preprints, Working Papers, ... Year : 2024

Paving the way toward foundation models for irregular and unaligned Satellite Image Time Series

Iris Dumeur
Silvia Valero
  • Function : Author
  • PersonId : 937533
Jordi Inglada

Abstract

Although recently several foundation models for satellite remote sensing imagery have been proposed, they fail to address major challenges of real/operational applications. Indeed, embeddings that don’t take into account the spectral, spatial and temporal dimensions of the data as well as the irregular or unaligned temporal sampling are of little use for most real world uses. As a consequence, we propose an ALIgned Sits Encoder (ALISE), a novel approach that leverages the spatial, spectral, and temporal dimensions of irregular and unaligned SITS while producing aligned latent representations. Unlike SSL models currently available for SITS, ALISE incorporates a flexible query mechanism to project the SITS into a common and learned temporal projection space. Additionally, thanks to a multi-view framework, we explore integration of instance discrimination along a masked autoencoding task to SITS. The quality of the produced representation is assessed through three downstream tasks: crop segmentation (PASTIS), land cover segmentation (MultiSenGE), and a novel crop change detection dataset. Furthermore, the change detection task is performed without supervision. The results suggest that the use of aligned representations is more effective than previous SSL methods for linear probing segmentation tasks.
Fichier principal
Vignette du fichier
article_v2.pdf (5.15 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04639033 , version 1 (11-07-2024)
hal-04639033 , version 2 (27-09-2024)

Identifiers

Cite

Iris Dumeur, Silvia Valero, Jordi Inglada. Paving the way toward foundation models for irregular and unaligned Satellite Image Time Series. 2024. ⟨hal-04639033v2⟩
173 View
43 Download

Altmetric

Share

More