We show that the chronological removal of cycles from a walk on a graph, known as Lawler's loop-erasing procedure, generates a preLie co-algebra on the vector space spanned by the walks. In addition, we prove that the tensor and symmetric algebras of graph walks are graded Hopf algebras, provide their antipodes explicitly and recover the preLie co-algebra from a brace coalgebra on the tensor algebra of graph walks. Finally we exhibit sub-Hopf algebras associated to particular types of walks.