Asymptotic Distributions associated to Oja's Learning Equation for Neural Networks - Archive ouverte HAL
Article Dans Une Revue IEEE Transactions on Neural Networks Année : 1999

Asymptotic Distributions associated to Oja's Learning Equation for Neural Networks

Résumé

In this paper, we perform a complete asymptotic performance analysis of the stochastic approximation algorithm (denoted Subspace Network Learning algorithm) derived from Oja's learning equation, in the case where the learning rate is constant and a large number of patterns is available. This algorithm drives the connection weight matrix W to an orthonormal basis of a dominant invariant subspace of a covariance matrix. Our approach consists in associating to this algorithm a second stochastic approximation algorithm that governs the evolution of WW T to the projection matrix onto this dominant invariant subspace. Then, using a general result of Gaussian approximation theory, we derive the asymptotic distribution of the estimated projection matrix. Closed form expressions of the asymptotic covariance of the projection matrix estimated by the SNL algorithm, and by the smoothed SNL algorithm that we introduce, are given in case of independent or correlated learning patterns and are further analyzed. It is found that the structures of these asymptotic covariance matrices are similar to those describing batch estimation techniques. The accuracy or our asymptotic analysis is checked by numerical simulations and it is found to be valid not only for a "small" learning rate but in a very large domain. Finally, improvements brought by our smoothed SNL algorithm are shown, such as the learning speed/misadjustment tradeoff and the deviation from orthonormality.
Fichier principal
Vignette du fichier
6.pdf (554.99 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03435749 , version 1 (18-11-2021)

Identifiants

  • HAL Id : hal-03435749 , version 1

Citer

Jean-Pierre Delmas, Jean-François Cardoso. Asymptotic Distributions associated to Oja's Learning Equation for Neural Networks. IEEE Transactions on Neural Networks, 1999. ⟨hal-03435749⟩
18 Consultations
35 Téléchargements

Partager

More