We present an automatic tool for tracing the dynamic of the smile intensity along a video record. The processed output consist in a sequence of adjusted time intervals labeled following the Smiling Intensity Scale of Gironzetti et al. (2016) [3], a 5 levels scale varying from neutral facial expression to laughing smile. The state-of-the-art toolbox OpenFace [2] is firstly used for tracking the face and for measuring the intensities of the facial Action Units of interest all along the video. In a second step the smile intensity automatic annotation is performed based on these OpenFace Action Units measurements. The statistical model underlying our SMAD tool is trained on a 1 hour manually annotated smiles of the CHEESE! corpus [4] (a full description of the model will be found in Rauzy & Amoyal, submitted to JMUI).
The evaluation of the engine reveals an observed agreement of 68% between manual and automatic annotations. A more concrete experiment conducted on in-the-wild video records shows that manually correcting the labels and interval boundaries of the automatic outputs reduces by a factor 10 the annotation time as compared with the time spent for manually annotating smile intensities without pretreatment. The smile annotation of PACO [1], a 5 hours corpus of conversational data built up for analyzing the impact of common ground in spontaneous face-to-face interaction, has already benefited from this gain in annotation time.
The SMAD scripts and documentation are available to download at the HMAD open source project url page https : //github:com/srauzy/HMAD.

References:
[1] Amoyal M, Priego-Valverde B, Rauzy S (2020) PACO : A corpus to analyze the impact of common ground in spontaneous face-to-face interaction. In: Language Resources and Evaluation Conference, LREC 2020, Marseille, France
[2] Baltruisaitis T, Zadeh A, Lim YC, Morency LP (2018) Openface 2.0: Facial behavior analysis toolkit. In: 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), pp 59-66
[3] Gironzetti E, Attardo S, Pickering L (2016) Smiling, gaze, and humor in conversation: A pilot study. In: Ruiz-Gurillo L (ed) Metapragmatics of Humor: Current research trends, pp 235-254
[4] Priego-Valverde B, Bigi B, Attardo S, Pickering L, Gironzetti E (2018) Is smiling during humor so obvious? A cross-cultural comparison of smiling behavior in humorous sequences in American English and French interactions. Intercultural Pragmatics