Transfer Learning in Computer Vision Tasks: Remember Where You Come From - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Image and Vision Computing Année : 2020

Transfer Learning in Computer Vision Tasks: Remember Where You Come From

Résumé

Fine-tuning pre-trained deep networks is a practical way of benefiting from the representation learned on a large database while having relatively few examples to train a model. This adjustment is nowadays routinely performed so as to benefit of the latest improvements of convolutional neural networks trained on large databases. Fine-tuning requires some form of reg-ularization, which is typically implemented by weight decay that drives the network parameters towards zero. This choice conflicts with the motivation for fine-tuning, as starting from a pre-trained solution aims at taking advantage of the previously acquired knowledge. Hence, regularizers promoting an explicit inductive bias towards the pre-trained model have been recently proposed. This paper demonstrates the versatility of this type of regular-izer across transfer learning scenarios. We replicated experiments on three state-of-the-art approaches in image classification, image segmentation, and video analysis to compare the relative merits of regularizers. These tests show systematic improvements compared to weight decay. Our experimental protocol put forward the versatility of a regularizer that is easy to implement and to operate that we eventually recommend as the new baseline for future approaches to transfer learning relying on fine-tuning.
Fichier principal
Vignette du fichier
IVC_Li2020.pdf (614.93 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02988362 , version 1 (04-11-2020)

Identifiants

Citer

Xuhong Li, Yves Grandvalet, Franck Davoine, Jingchun Cheng, Yin Cui, et al.. Transfer Learning in Computer Vision Tasks: Remember Where You Come From. Image and Vision Computing, 2020, 93, pp.103853. ⟨10.1016/j.imavis.2019.103853⟩. ⟨hal-02988362⟩
136 Consultations
284 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More