Unrolled optimization methods have emerged as a way to combine classical iterative optimization techniques with learned priors to efficiently solve image restoration problems. However, learning the regularization prior along the unrolled iterations requires intensive memory usage due to the deep explicit backpropagation, hence making the number of unrolled iterations usually small in practice. Inspired by deep equilibrium models, unrolling models with implicit backpropagation have been considered for solving this issue. Nevertheless, while these methods yield good restoration quality with reduced memory usage, the theory of implicit backpropagation relies on the knowledge of the fixed point of the function to optimize, usually unknown and estimated by iterating until convergence. Therefore, these methods require intensive computation time to ensure a stable backpropagation. In this paper, we present an unrolled Proximal Point Algorithm method, where the end-toend optimization problem is redefined as a per unrolled iteration optimization problem. We prove that the proposed optimization strategy is memory-efficient and applicable for any number of computed unrolled iteration. We empirically show that our method achieves state-of-the-art image restoration quality.