Co-production of speech and pointing gestures in clear and perturbed interactive tasks: multimodal designation strategies - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2017

Co-production of speech and pointing gestures in clear and perturbed interactive tasks: multimodal designation strategies

Résumé

Designation consists in attracting an interlocutor's attention on a specific object and/or location. It is most often achieved using both speech (e.g., demonstratives) and gestures (e.g., manual pointing). This study aims at analyzing how speech and pointing gestures are co-produced in a semi-directed interactive task involving designation. 20 native speakers of French were involved in a cooperative task in which they provided instructions to a partner for her to reproduce a model she could not see on a grid both of them saw. They had to use only sentences of the form 'The [target word] goes there.'. They did this in two conditions: silence and noise. Their speech and articulatory/hand movements (motion capture) were recorded. The analyses show that the participants' speech features were modified in noise (Lombard effect). They also spoke slower and made more pauses and errors. Their pointing gestures lasted longer and started later showing an adaptation of gesture production to speech. The condition did not influence speech/gesture coordination. The apex (part of the gesture that shows) mainly occurred at the same time as the target word and not as the demonstrative showing that speakers group speech and gesture carrying complementary rather than redundant information.
Fichier principal
Vignette du fichier
DohenRoustan_IS2017.pdf (986.5 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02367749 , version 1 (18-11-2019)

Identifiants

  • HAL Id : hal-02367749 , version 1

Citer

Marion Dohen, Benjamin Roustan. Co-production of speech and pointing gestures in clear and perturbed interactive tasks: multimodal designation strategies. Interspeech 2017 - 18th Annual Conference of the International Speech Communication Association, Aug 2017, Stockholm, Sweden. ⟨hal-02367749⟩
118 Consultations
167 Téléchargements

Partager

Gmail Facebook X LinkedIn More