Online compensation of auditory feedback perturbations in speech: an optimalfeedback control model of tongue biomechanics - Archive ouverte HAL
Communication Dans Un Congrès Année : 2024

Online compensation of auditory feedback perturbations in speech: an optimalfeedback control model of tongue biomechanics

Résumé

Introduction. Online responses to perturbations of the auditory feedback have been extensively studied since the pioneering study of Houde and Jordan (1998). A striking finding is that subjects tend to rely preferentially on either the somatosensory or the auditory feedback to adjust speech production (Lametti, Nasir, and Ostry 2012). Another remarkable phenomenon is that even modest online compensation tend to show an asymmetry between up- and down-shifts of the speech formants (Hantzsch, Parrell, and Niziolek 2022). Here we present a modeling analysis of these experiments using GEPPETO-OC This model integrates the neurobiomechanical model of GEPPETO (Payan and Perrier 1997) into a stochastic feedback control framework (Todorov and Jordan 2002), in order to handle high level feedback control during movement execution. This new framework also allows to account for the impact of motor and sensory variabilitywhile preserving the idea of optimal effort allocation. We here explore whether the interaction between biomechanical constraints and characteristics of the sensory feedbacks can explain human online audiomotor control.Methods. Speech goals were considered multimodal: auditory (target region for phonemes defined as ellipsoids in F1/F2 and F3/F2 domains) and proprioceptive (ellipsoids in the 3D space of variables that characterize the tongue shape). For simplicity, the timing of these goal sequence was considered fixed.GEPPETO-OC does not separate planning and execution: in order to determine the current motor commands, the optimal controller computed and adapted the trajectories of the motor commands in real time by constantly minimizing both the neuromuscular effort and the sensory inaccuracy at the phonemic targets as a hybrid cost.Motor and sensory signals were assumed corrupted by additive and multiplicative noises, and feedback was assumed delayed by up to 80 ms. An optimal estimator (EKF) estimated the state of the system through an internal model of the biomechanics based on an efferent copy of the motor commands, predicted the sensory reafference, and corrected the state estimate as a function of the sensory prediction error. The original GEPPETO model controlled a finite-element biomechanical model of the tongue. In GEPPETO-OC, to speed up computation, a reduced model was developed. First, the dimensionality of the upper contour of the tongue, described by the position of 16 nodes in the sagittal plane, was reduced to 5D via an autoencoder. Then a LSTM network model of the plant dynamics was trained on thousands of simulations of the finite element model. Last, auditory feedback was computed from the tongue contour using a harmonic model of the vocal tract (Badin and Fant 1984).Results. To assess whether across-subject differences in sensory precision could explain differential reliance on somatosensory or auditory feedback, we simulated trajectories aimed at the /E/ target from a neutral tongue position in three different conditions: (1) no auditory perturbation; called reference on Figure 1; (2) the auditory feedback of F1 was up-shifted by 125 mels from the onset of the movement; (3) F1 was conversely down-shifted by 125 mels. For these three conditions, we tested two different cases: (a) the auditory signal was more noisy than the somatosensory signal; (b) the somatosensory signal was more noisy than the auditory signal. Each condition was repeated 20 times to compare trial-to-trial variability.In the (a) case (strong auditory noise), compensation value at the end of the movement is less than 10 mels ( 8%) in upward and downward shift. In the (b) case, compensation value reach approximately 35 mels ( 28%). The dynamics of the compensation were also different between (a) and (b), case (a) being close to linear. Compensation of the up-shift and down-shift were also asymmetrical, but differently so in the two cases studied.Discussion. Our model could reproduce the sensory preference effects by only assuming a difference in sensory precision. Furthermore, the predicted compensation was modest and larger for down-shifts in setting (a), as found experimentally (Hantzsch, Parrell, and Niziolek 2022); this result is likely due to concurrent optimization of precision and effort. In the "auditory preference" case (b), the dynamics of the compensation clearly reflected the asymmetry between an assistive and a resistive perturbation.Results could also be analyzed in the kinematic domain (tongue contour motion) and in the motor domain (change in muscle activation patterns). This additional data will be presented during the conference.Future work should compare the effect of a perturbation on either tongue movement (as here) or on stable vowels (static articulatory positions). This would check whether the model is also able to reproduce the experimental observations of a much slower compensation (at around 460 ms) under the latter experimental condition.
Fichier principal
Vignette du fichier
Abstract_Template_for_ISSP_2024 (1).pdf (261.74 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04738746 , version 1 (15-10-2024)

Identifiants

  • HAL Id : hal-04738746 , version 1

Citer

Tsiky Rakotomalala, Pierre Baraduc, Pascal Perrier. Online compensation of auditory feedback perturbations in speech: an optimalfeedback control model of tongue biomechanics. ISSP 2024 - 13th International Seminar on Speech Production, May 2024, Autrans (Grenoble), France. ⟨hal-04738746⟩
77 Consultations
15 Téléchargements

Partager

More