Different orientations of rays emitted by a 3D scene are captured in Light field (LF) imaging. In this, spatial along with angular information of the scene is captured. LF offers higher dimensional representation which needs advanced processing such as post-capture refocusing, depth sensing, compression, and reconstruction. These processing methods often induce artifacts as the quality is degraded. So, it is important to understand the mechanism of human visual quality perception to find the optimal performance that maximizes the quality of experience (QoE) of LF. The measurement of this data degradation is possible with the availability of a suitable dataset. In this paper, we present a comprehensive overview and discussion of publicly available LF datasets with emphasis on dataset with subject score. Furthermore, we discuss the current trends and existing challenges in LF dataset creation for QoE.