We propose to tackle the disocclusion inpainting of RGB-D images when synthesizing new views of a scene by changing its viewpoint. Such a process creates holes both in depth and color images. First, we propose a novel algorithm to perform the depth-map disocclusion inpainting. In particular, this intuitive approach is able to recover the lost structures of the objects and to inpaint the depth-map in a geometrically plausible manner. Then, we propose a depth-guided patch based inpainting method to fill-in the color image. Depth information coming from the reconstructed depth-map is added to each key step of the classical patch-based algorithm from Criminisi et al. in an intuitive manner. Relevant comparisons to state-of-the-art inpainting methods for the disocclusion inpainting of both depth and color images are provided and illustrate the effectiveness of our proposed algorithms.