Persistence, defined as the probability that a signal has not reached a threshold up to a given observation time, plays a crucial role in the theory of random processes. Often, persistence decays algebraically with time with non trivial exponents. However, general analytical methods to calculate persistence exponents cannot be applied to the ubiquitous case of non-Markovian systems relaxing transiently after an imposed initial perturbation. Here, we introduce a theoretical framework that enables the non-perturbative determination of persistence exponents of Gaussian non-Markovian processes with non stationary dynamics relaxing to a steady state after an initial perturbation. Two situations are analyzed: either the system is subjected to a temperature quench at initial time, or its past trajectory is assumed to have been observed and thus known. Our theory covers the case of spatial dimension higher than one, opening the way to characterize non-trivial reaction kinetics for complex systems with non-equilibrium initial conditions.