Deep neural networks are difficult to train when applied to tasks that can be expressed as algorithmic procedures. In this article, we propose to study how the explicit guidance of a network through all steps of the algorithm, using external memory and active choice of inputs, can improve its learning capability. The idea is to take inspiration from a child's learning and running through a procedure via interaction with an external support such as a paper. We show that this mechanism applied to a simple multilayer perceptron can significantly improve its performance when learning either a multi-digit addition or multiplication, which are simple but yet challenging operations to learn.