Note on Backpropagation in Neural Networks
Résumé
This note intends to facilitate low level implementation by providing an analytical perspective on neural networks. Different feedforward and recurrent neural networks are dissected through a derivation of the backpropagation update. We choose Multilayer Perceptron (MLP) which possesses the basic architecture of deep artificial neural network as a departure of introduction. Sigmoid Cross-Entropy loss is applied to MLP for an exemplification of multi-label classification. We then turn to introduce Convolutional Neural Network (CNN)-an intricate architecture which adopts filter and sub-sampling to realize a form of regularization. In the end, we illustrate Backpropagation Through Time (BPTT) to elicit Exploding / Vanishing Gradients problem and Long short-term memory (LSTM).
Domaines
Intelligence artificielle [cs.AI]
Origine : Fichiers produits par l'(les) auteur(s)
Loading...