Embedding in Neural Networks: A-Priori Design of Hybrid Computers for Prediction
Résumé
The prediction of complex signals is among the most important
applications of random recurrent Neural Networks (rRNN). Yet, no theory
which completely describes prediction in rRNNs exists. As such, these
systems remain "black boxes". Based on nearest neighbors theory and
random nonlinear mapping, we fully describe the mechanisms employed by
rRNNs solving this essential task. Our approach combines machine
learning techniques (Reservoir Computing) and dynamical systems theory.
We derive optimization cost functions which are (a) task specific, and
(b) go far beyond the simple optimization of the prediction error. Going
beyond, we demonstrate the consequences resulting from our theory.
Based on our analysis of an rRNN stabilizing of an arrhythmic heart, we
amend this system by a nonvolatile external FIFO memory. Furthermore, we
are able a-priori specify the FIFO's properties, strongly reducing the
previous training efforts of such systems.