On the inductive biases of deep domain adaptation - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

On the inductive biases of deep domain adaptation

Rodrigue Siry
  • Fonction : Auteur
  • PersonId : 1105723
Loïc Simon

Résumé

Even if it is not always stated explicitly, the majority of recent approaches to domain adaptation are based on theoretically unjustified assumptions, on the one hand, and (often hidden) inductive biases on the other. This paper first point out that {\em feature alignment} which is often misrepresented as a minimizer of some theoretical upper-bounds on risk in the target domain is neither necessary nor sufficient to obtain low target risk. This paper also demonstrates, through numerous experiments, that deep domain adaptation methods, in fact, rely heavily on some hidden inductive biases found in common practices, such as model pretraining or encoder architecture design. In a third step, the paper argues that using handcrafted priors might not be sufficient to bridge distant domains: powerful parametric priors can be instead learned from data, leading to a large improvement in target accuracy. A meta-learning strategy allowing to find inductive biases that solve specific transfers is proposed. It shows superior performance to that of handcrafted priors on several tasks.
Fichier principal
Vignette du fichier
Inductive_Biases_of_domain_adaptation____cviu.pdf (432.38 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03290701 , version 1 (19-07-2021)
hal-03290701 , version 2 (25-04-2023)

Identifiants

  • HAL Id : hal-03290701 , version 2

Citer

Rodrigue Siry, Louis Hémadou, Loïc Simon, Frédéric Jurie. On the inductive biases of deep domain adaptation. 2023. ⟨hal-03290701v2⟩
71 Consultations
211 Téléchargements

Partager

Gmail Facebook X LinkedIn More