Exploration of dynamic natural scenes: influence of unrelated soundtracks on eye movements
Résumé
When exploring audio-visual scenes, our brain integrates information from both modalities. So far, attempts to understand audio-visual integration mostly used artificial stimuli. In a previous study, we showed that removing the soundtrack of natural dynamic scenes impacts on their exploration [Coutrot, A. et al. (2012). Influence of soundtrack on eye movements during video exploration. Journal of Eye Movement Research, 5(4)]. Here, we refine this work by studying the effect of a specific auditory scene on the exploration of a specific visual scene. We built a database of 45 videos split into 3 visual classes (Faces, Moving Objects and Landscapes). We eye-tracked 60 participants viewing each video in one of the following conditions: with its original soundtrack (Speech for Faces, Object Noises for Moving Objects, Environmental Noises for Landscapes); with a soundtrack of a video belonging to the same visual class; with a soundtrack of a video belonging to another visual class. We show that for Moving Objects and Landscapes, changing the auditory condition does not have a significant effect on eye movements. For Faces, we observe significant differences between the "original soundtrack" condition and the other auditory conditions. This might traduce the cognitive specificity of speech perception.