Time to augment self-supervised visual representation learning - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Time to augment self-supervised visual representation learning

Markus R Ernst
  • Fonction : Auteur
  • PersonId : 1364164
Céline Teulière
Jochen Triesch
  • Fonction : Auteur
  • PersonId : 1036139

Résumé

Biological vision systems are unparalleled in their ability to learn visual representations without supervision. In machine learning, self-supervised learning (SSL) has led to major advances in forming object representations in an unsupervised fashion. Such systems learn representations invariant to augmentation operations over images, like cropping or flipping. In contrast, biological vision systems exploit the temporal structure of the visual experience during natural interactions with objects. This gives access to “augmentations” not commonly used in SSL, like watching the same object from multiple viewpoints or against different backgrounds. Here, we systematically investigate and compare the potential benefits of such time-based augmentations during natural interactions for learning object categories. Our results show that incorporating time-based augmentations achieves large performance gains over state-of-the-art image augmentations. Specifically, our analyses reveal that: 1) 3-D object manipulations drastically improve the learning of object categories; 2) viewing objects against changing backgrounds is important for learning to discard background-related information from the latent representation. Overall, we conclude that time-based augmentations during natural interactions with objects can substantially improve self-supervised learning, narrowing the gap between artificial and biological vision systems.
Fichier non déposé

Dates et versions

hal-04506217 , version 1 (15-03-2024)

Identifiants

  • HAL Id : hal-04506217 , version 1

Citer

Arthur Aubret, Markus R Ernst, Céline Teulière, Jochen Triesch. Time to augment self-supervised visual representation learning. The Eleventh International Conference on Learning Representations (ICLR), May 2023, Kigali, France. ⟨hal-04506217⟩
0 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More