We extend the theory of cellular automata to arbitrary, time-varying graphs. In other words we formalise, and prove theorems about, the intuitive idea of a labelled graph which evolves in time -- but under the natural constraint that information can only ever be transmitted at a bounded speed, with respect to the distance given by the graph. The notion of translation-invariance is also generalised. The definition we provide for these 'causal graph dynamics' is simple and axiomatic. The theorems we provide also show that it is robust. For instance, causal graph dynamics are stable under composition and under restriction to radius one. In the finite case some fundamental facts of cellular automata theory carry through: causal graph dynamics admit a characterisation as continuous functions, and they are stable under inversion. The provided examples suggest a wide range of applications of this mathematical object, from complex systems science to theoretical physics.