A classical random walk (S t , t ∈ N) is defined by S t := t n=0 X n , where (X n) are i.i.d. When the increments (X n) n∈N are a one-order Markov chain, a short memory is introduced in the dynamics of (S t). This so-called " persistent " random walk is nolonger Markovian and, under suitable conditions, the rescaled process converges towards the integrated telegraph noise (ITN) as the timescale and space-scale parameters tend to zero (see [11, 17, 18]). The ITN process is effectively non-Markovian too. The aim is to consider persistent random walks (S t) whose increments are Markov chains with variable order which can be infinite. This variable memory is enlighted by a one-to-one correspondence between (X n) and a suitable Variable Length Markov Chain (VLMC), since for a VLMC the dependency from the past can be unbounded. The key fact is to consider the non Markovian letter process (X n) as the margin of a couple (X n , M n) n≥0 where (M n) n≥0 stands for the memory of the process (X n). We prove that, under a suitable rescaling, (S n , X n , M n) converges in distribution towards a time continuous process (S 0 (t), X(t), M (t)). The process (S 0 (t)) is a semi-Markov and Piecewise Deterministic Markov Process whose paths are piecewise linear.