Training Vision Transformers for Image Retrieval - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2022

Training Vision Transformers for Image Retrieval

Résumé

Transformers have shown outstanding results for natural language understanding and, more recently, for image classification. We here extend this work and propose a transformer-based approach for image retrieval: we adopt vision transformers for generating image descriptors and train the resulting model with a metric learning objective, which combines a contrastive loss with a differential entropy regularizer. Our results show consistent and significant improvements of transformers over convolutionbased approaches. In particular, our method outperforms the state of the art on several public benchmarks for category-level retrieval, namely Stanford Online Product, In-Shop and CUB-200. Furthermore, our experiments on ROxford and RParis also show that, in comparable settings, transformers are competitive for particular object retrieval, especially in the regime of short vector representations and low-resolution images.
Fichier principal
Vignette du fichier
irt_haltools.pdf (3.37 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03572734 , version 1 (14-02-2022)

Identifiants

  • HAL Id : hal-03572734 , version 1

Citer

Alaaeldin El-Nouby, Natalia Neverova, Ivan Laptev, Hervé Jégou. Training Vision Transformers for Image Retrieval. 2022. ⟨hal-03572734⟩
121 Consultations
249 Téléchargements

Partager

Gmail Facebook X LinkedIn More