Article Dans Une Revue Computer Vision and Image Understanding Année : 2023

On the inductive biases of deep domain adaptation

Résumé

Even if it is not always stated explicitly, the majority of recent approaches to domain adaptation are based on theoretically unjustified assumptions, on the one hand, and (often hidden) inductive biases on the other. This paper first point out that {\em feature alignment} which is often misrepresented as a minimizer of some theoretical upper-bounds on risk in the target domain is neither necessary nor sufficient to obtain low target risk. This paper also demonstrates, through numerous experiments, that deep domain adaptation methods, in fact, rely heavily on some hidden inductive biases found in common practices, such as model pretraining or encoder architecture design. In a third step, the paper argues that using handcrafted priors might not be sufficient to bridge distant domains: powerful parametric priors can be instead learned from data, leading to a large improvement in target accuracy. A meta-learning strategy allowing to find inductive biases that solve specific transfers is proposed. It shows superior performance to that of handcrafted priors on several tasks.
Fichier principal
Vignette du fichier
Inductive_Biases_of_domain_adaptation____cviu.pdf (432.38 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03290701 , version 1 (19-07-2021)
hal-03290701 , version 2 (25-04-2023)

Identifiants

Citer

Rodrigue Siry, Louis Hémadou, Loïc Simon, Frédéric Jurie. On the inductive biases of deep domain adaptation. Computer Vision and Image Understanding, 2023, 233, pp.103714. ⟨10.1016/j.cviu.2023.103714⟩. ⟨hal-03290701v2⟩
108 Consultations
263 Téléchargements

Altmetric

Partager

More