Holography is often considered as the most promising immersive technology because it provides all the depth cues of the human visual system. Some limitations still need to be overcome such as the huge computational load of high-definition holograms and the noise introduced in the reconstructed scene during the quantization process. In this paper, we propose what we believe is a novel view-specific layer-based stereogram approach combined with a view-dependent error diffusion algorithm that aims to solve those limitations. This method selects the light waves of the 3D scene that reach a specific viewing area and leverages this particular configuration to apply an error diffusion algorithm. Two additional quality enhancement features are observed: the reduction of the conjugate order perceptibility and the increased brightness of the reconstructed scene. Numerical and optical experiments demonstrate the time savings and quality enhancements of our approach.