On the inductive biases of deep domain adaptation
Résumé
Even if it is not always stated explicitly, the majority of recent approaches to domain adaptation are based on theoretically unjustified assumptions, on the one hand, and (often hidden) inductive biases on the other. This paper first point out that {\em feature alignment} which is often misrepresented as a minimizer of some theoretical upper-bounds on risk in the target domain is neither necessary nor sufficient to obtain low target risk. This paper also demonstrates, through numerous experiments, that deep domain adaptation methods, in fact, rely heavily on some hidden inductive biases found in common practices, such as model pretraining or encoder architecture design.
In a third step, the paper argues that using handcrafted priors might not be sufficient to bridge distant domains: powerful parametric priors can be instead learned from data, leading to a large improvement in target accuracy. A meta-learning strategy allowing to find inductive biases that solve specific transfers is proposed. It shows superior performance to that of handcrafted priors on several tasks.
Domaines
Intelligence artificielle [cs.AI]
Fichier principal
Inductive_Biases_of_domain_adaptation____cviu.pdf (432.38 Ko)
Télécharger le fichier
Origine | Fichiers produits par l'(les) auteur(s) |
---|