Here we report that large pupils predict fixations of the eye on low-salient, inconspicuous parts of a visual scene. We interpret this as showing that mental effort, reflected by a dilation of the pupil, is required to guide gaze toward objects that are relevant to current goals, but that may not be very salient. When mental effort is low, reflected by a constriction of the pupil, the eyes tend to be captured by high-salient parts of the image, irrespective of top-down goals. The relationship between pupil size and visual saliency was not driven by luminance or a range of other factors that we considered. Crucially, the relationship was strongest when mental effort was invested exclusively in eye-movement control (i.e., reduced in a dual-task setting), which suggests that it is not due to general effort or arousal. Our finding illustrates that goal-driven control during scene viewing requires mental effort, and that pupil size can be used as an online measure to track the goal-drivenness of behavior.