Do image properties drive first ocular explorations of natural scene during free viewing? The response is positive, according to previous works defining image properties with statistical measures or modeled with a saliency map. But, because these works were mainly based on correlation analysis (but see Einhauser, 2002), no causal link could be established between image properties and ocular fixations. Therefore, across three experiments (180 subjects) we manipulated the saliency and semantic congruency of small regions within 72 natural scenes during a categorization task. In the 'saliency' experiment, the mean saliency of one region was reduced for half the scenes. In the 'semantic' experiment, an incongruent object (object with a very low probability of occurrence in the scene context) was introduced. This object was substituted for another but without any modification in the saliency. For both experiments the luminance distribution of the modified regions were held constant. The third experiment was a control condition (without any manipulation). Eye movements were recorded during the 3s of scene presentation. The number of fixations and the time spent inside the manipulated regions were calculated. In contrast with control condition, data analysis showed that, first, when the saliency of one region was reduced the region was less attractive (the number of fixations and the time spent inside the region were lower than in the control condition). Secondly, when an incongruent object was inserted, in contrast to the expectations of the literature (Henderson & Hollingworth, 1998), the region was less attractive. In conclusion, on one hand; we establish a causal link between saliency and ocular fixations. On the other hand; keeping most of image properties constant, we show that ocular fixations are flexibly driven by the congruency of an object and its context. Incongruent object were less attractive during the first moment of exploration, in contrary to longer exploration.