The parallel replica dynamics method, originally developed by A.F. Voter, can efficiently simulate very long trajectories of Langevin dynamics in the presence of metastability. In this work, we extend the algorithm to discrete in time Markov processes. We give a mathematical analysis of the algorithm which relies on the properties of quasistationary distributions. One application of the algorithm is the correction of time step discretization errors for the original parallel replica dynamics applied to Langevin dynamics.