In this paper, we present a new formalism for nonlinear and non-separable multi-scale representations. We first show that most of the one-dimensional nonlinear multi-scale representations described in the literature are based on prediction operators which are the sum of a linear prediction operator and a perturbation defined using finite differences. We then extend this point of view to the multi-dimensional case where the scaling factor is replaced by a non-diagonal dilation matrix $M$. The new formalism we propose brings about similarities between existing nonlinear multi-scale representations and also enables us to alleviate the classical hypotheses made to prove the convergence of the multi-scale representations.