Hearing gestures and drawing sounds: Auditory and multisensory perception of biological movements
Résumé
Understanding the links between the acoustical properties of sounds produced by our actions and the properties of
the action imagined from them is a challenge that may help to understand how our sensorimotor interaction with the
surrounding world constrains our perception and reciprocally. Here we’ll present results of recent works on the
acoustic characterization of a particular category of biological movements, namely drawing movements. If we listen
carefully to the friction sounds produced by someone drawing, a movement can be perceived. How do we interpret
this movement perceived from a friction sound? Is it for instance possible to recognize whether or not it is a human
movement and can the drawn shapes be deduced from the dynamics of such sounds? To investigate these
questions, we developed sound synthesis tools enabling the generation of virtual friction sounds which can be
precisely calibrated according to movements parameters such as the velocity. A first experiment showed that we are
able to recognized biological drawing movements characterized by the 2/3 power law through kinematic related
friction sounds (Viviani et al., JEP:HPP 1992, Thoret et al., JEP:HPP 2014). In a second time, we showed that
kinematic related friction sounds enable the recognition of geometrical shapes and even to characterize the prototype
(Rosch, 1973) of an elliptical shape. Finally, in a multimodal context, we showed that friction sounds evoking
movements may distort the visuo-motor coupling of drawing movements (Thoret et al., PLoS one, 2016). Taken
together, all these experiments reveal the role of the auditory perception in the perception and production of biological
movements. Moreover, it provides a new framework to investigate the relationship between sounds and movements.