The use of silent mouth openings to show upcoming intervention in triadic screen mediated gaming interactions
Résumé
The human face plays an important role in communication as facial expressions can convey various information to frame the verbal turn they accompany, precede or follow. They can also display how a co-participant's current verbal turn (or other action) is perceived and/or understood (e.g. Gudmundsen & Svennevig, 2020). Facial expressions also play a significant role in the systematics of turn taking (e.g. Duncan & Niederehe, 1974) but have yet been far less analysed in detail than other bodily behaviour such as changes in posture and head movements.
Our data come from three party interactions in a collaborative digital game where the players can communicate via their webcam image representations. This 'talking head' format (Morel & Licoppe, 2009) gives only partial access to a participant's body. As the most visible part of the player is their head, mimics represent a communication device that is accessible with the least possible effort (since the head is by default centred in the webcam frame, whereas for example hand gestures have to be realised in a particular, and not always natural, zone in order to make them visible for the other participants). In the context of this collaborative game, the players need to jointly make decisions to achieve their shared goal. Thus, while discussing and arguing, they constantly need to display their (dis)agreement. The triadic interaction is regularly organised around dyadic exchange which the third participant has to find their 'way back' into.
In the present talk, I will show one particular way of 'coming back into a discussion' in which turn-initial facial movements play an essential role: the player who tries to bring themselves into the discussion comes to a rather neutral (relaxed) facial expression, which can express concentrated listening. Reacting to a prior turn, they either ask for more information that is needed to be able to do a particular action and that had been expected to be given by the co-participants, or they give additional information on a topic at hand which could influence the decision to make on it. To prepare or announce their upcoming intervention, they open their mouth, sometimes in combination with audible inbreath, but due to the screen mediated interaction context, breathing is not always audible to the co-participants.
By conducting a sequential analysis of three examples, chosen from a corpus of six different groups of players, I will show how this mouth opening followed by a speech fragment is sequentially treated and allows the player to make themselves heard.
Duncan, S., & Niederehe, G. (1974). On Signalling That It’s Your Turn to Speak. Journal of Experimental Social Psychology, 10, 234–247. https://doi.org/10.1016/0022-1031(74)90070-5
Gudmundsen, J., & Svennevig, J. (2020). Multimodal displays of understanding in vocabulary-oriented sequences. Social Interaction. Video-Based Studies of Human Sociality, 3(2). https://doi.org/10.7146/si.v3i2.114992
Morel, J., & Licoppe, C. (2009). La vidéocommunication sur téléphone mobile—Quelle mobilité pour quels cadrages ? Réseaux, 4(n° 156), 165–201.