Robots need to identify environmental cues like humans do for effective human-robot interaction (HRI). Human attention models simulate the way humans process visual information, making them useful for identifying important regions in images/videos. In this paper, we explore the use of human attention models in developing intuitive and anthropomorphic HRI. Our approach combines a saliency model and a moving object detection model. The framework is implemented using the ROS framework on Pepper, a humanoid robot. To evaluate the effectiveness of our system, we conducted both subjective and qualitative measures, including subjective rating measures to evaluate intuitiveness, trust, engagement, and user satisfaction, and quantitative measures of our human attention subsystem against state-of-the-art models. Our extensive experiments demonstrate the significant impact of our framework in enabling intuitive and anthropomorphic human-robot interaction.