After reviewing the behavioral studies of working memory and of its cellular substrate, we argue that metastable states constitute candidates for the type of transient information storage required by working memory. We then present a simple neural network model made of stochastic units whose synapses exhibit short-term facilitation. This model was specifically designed to be analytically tractable, simple to simulate numerically and to exhibit a strong form of metastability. We present both numerical results: the existence of metastable states able to represent scalar quantities; and analytical ones: the properties of the metastable states that can be fully deduced from an implicit equation once the 4 model parameters have been specified.