Implicit Regularization with Polynomial Growth in Deep Tensor Factorization - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Implicit Regularization with Polynomial Growth in Deep Tensor Factorization

Résumé

We study the implicit regularization effects of deep learning in tensor factorization. While implicit regularization in deep matrix and 'shallow' tensor factorization via linear and certain type of non-linear neural networks promotes low-rank solutions with at most quadratic growth, we show that its effect in deep tensor factorization grows polynomially with the depth of the network. This provides a remarkably faithful description of the observed experimental behaviour. Using numerical experiments, we demonstrate the benefits of this implicit regularization in yielding a more accurate estimation and better convergence properties.
Fichier principal
Vignette du fichier
hariz22a.pdf (1.53 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03726808 , version 1 (18-07-2022)

Identifiants

  • HAL Id : hal-03726808 , version 1

Citer

Kais Hariz, Hachem Kadri, Stéphane Ayache, Maher Moakher, Thierry Artières. Implicit Regularization with Polynomial Growth in Deep Tensor Factorization. International Conference on Machine Learning, Jul 2022, Baltimore, United States. ⟨hal-03726808⟩
148 Consultations
22 Téléchargements

Partager

Gmail Facebook X LinkedIn More