Some thoughts about transfer learning. What role for the source domain? - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue International Journal of Approximate Reasoning Année : 2024

Some thoughts about transfer learning. What role for the source domain?

Résumé

Transfer learning is called for when the training and test data do not share the same input distributions ($\Prob_\X^{\cal S} \neq \Prob_\X^{\cal T}$) or/and not the same conditional ones ($\Prob_{{\cal Y}|{\cal X}}^{\cal S} \neq \Prob_{{\cal Y}|{\cal X}}^{\cal T}$). In the most general case, the input spaces and/or output spaces can be different: $\X_{\cal S} \neq \X_{\cal T}$ and/or $\Y_{\cal S} \neq \Y_{\cal T}$. However, most work assume that $\X_{\cal S} = \X_{\cal T}$ . Furthermore, a common held assumption is that it is necessary that the source hypothesis be good on the source training data and that the ``distance'' between the source and the target domains be as small as possible in order to get a good (transferred) target hypothesis. This paper revisits the reasons for these beliefs and discusses the relevance of these conditions. An algorithm is presented which can deal with transfer learning problems where $\X_{\cal S} \neq \X_{\cal T}$, and that furthermore brings a fresh perspective on the role of the source hypothesis (it does not have to be good) and on what is important in the distance between the source and the target domains (translations between them should belong to a limited set). Experiments illustrate the properties of the method and confirm the theoretical analysis. Determining beforehand a relevant source hypothesis remains an open problem, but the vista provided here helps understanding its role.
Fichier principal
Vignette du fichier
2023_IJAR-v4-perso.pdf (2.47 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04488871 , version 1 (17-03-2024)

Identifiants

Citer

Antoine Cornuéjols. Some thoughts about transfer learning. What role for the source domain?. International Journal of Approximate Reasoning, 2024, 166, pp.109-146. ⟨10.1016/j.ijar.2023.109107⟩. ⟨hal-04488871⟩
0 Consultations
1 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More