In this paper, we consider the problem of vector-valued regularization of light fields based on PDEs. We propose a regularization method operating in the 4D ray space that does not require prior estimation of disparity maps. The method performs a PDE-based anisotropic diffusion along directions defined by local structures in the 4D ray space. We analyze light field regularization in the 4D ray space using the proposed 4D anisotropic diffusion framework by first considering a light field toy example, i.e., a tesseract. This simple light field example allows an in-depth analysis of how each eigenvector influences the diffusion process. We then illustrate the diffusion effect for several light field processing applications: denoising, angular and spatial interpolation, regularization for enhancing disparity estimation as well as inpainting.