We discuss coupling from the past techniques (CFTP) for perturbations of interacting particle systems on the d-dimensional integer lattice, with a finite set of states, within the framework of the graphical construction of the dynamics based on Poisson processes. We first develop general results for what we call CFTP times with ambiguities. These are analogous to classical coupling (from the past) times, except that the coupling property holds only provided that some ambiguities concerning the stochastic evolution of the system are resolved. If these ambiguities are rare enough on average, CFTP times with ambiguities can be used to build actual CFTP times, whose properties can be controlled in terms of those of the original CFTP time with ambiguities. We then prove a general perturbation result, which can be stated informally as follows. Start with an interacting particle system possessing a CFTP time whose definition involves the exploration of an exponentially integrable number of points in the graphical construction, and which satisfies the positive rates property. Then consider a perturbation obtained by adding new transitions to the original dynamics. Our result states that, provided that the perturbation is small enough (in the sense of small enough rates), the perturbed interacting particle system too possesses a CFTP time (with nice properties such as an exponentially decaying tail). The proof consists in defining a CFTP time with ambiguities for the perturbed dynamics, from the CFTP time for the unperturbed dynamics. Finally, we discuss examples of particle systems to which this result can be applied. Concrete examples include a class of neighbor-dependent nucleotide substitution model, and variations of the classical voter model, illustrating the ability of our approach to go beyond the case of weakly interacting particle systems.