Conference Papers Year : 2022

Local Identifiability of Deep ReLU Neural Networks: the Theory

Abstract

Is a sample rich enough to determine, at least locally, the parameters of a neural network? To answer this question, we introduce a new local parameterization of a given deep ReLU neural network by fixing the values of some of its weights. This allows us to define local lifting operators whose inverses are charts of a smooth manifold of a high dimensional space. The function implemented by the deep ReLU neural network composes the local lifting with a linear operator which depends on the sample. We derive from this convenient representation a geometrical necessary and sufficient condition of local identifiability. Looking at tangent spaces, the geometrical condition provides: 1/ a sharp and testable necessary condition of identifiability and 2/ a sharp and testable sufficient condition of local identifiability. The validity of the conditions can be tested numerically using backpropagation and matrix rank computations.
Fichier principal
Vignette du fichier
Local Identifiability of Deep ReLU Neural Networks the Theory (1).pdf (619) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03687395 , version 1 (14-06-2022)
hal-03687395 , version 2 (21-11-2022)

Licence

Identifiers

Cite

Joachim Bona-Pellissier, François Malgouyres, François Bachoc. Local Identifiability of Deep ReLU Neural Networks: the Theory. Advances in Neural Information Processing Systems, Nov 2022, New Orleans, United States. ⟨hal-03687395v2⟩
226 View
168 Download

Altmetric

Share

More