Towards multisensory control of physical modeling synthesis - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2024

Towards multisensory control of physical modeling synthesis

Résumé

Physical models of musical instruments offer an interesting tradeoff between computational efficiency and perceptual fidelity. Yet, they depend on a multidimensional space of user-defined parameters whose exploration by trial and error is impractical. Our article addresses this issue by combining two ideas: query by example and gestural control. On one hand, we train a deep neural network to identify the resonator parameters of a percussion synthesizer from a single audio example via an original method named perceptual–-neural–-physical sound matching (PNP). On the other hand, we map these parameters to knobs in a digital controller and configure a musical touchpad with MIDI polyphonic expression. Hence, we propose a multisensory interface between human and machine: it integrates haptic and sonic information and produces new sounds in real time as well as visual feedback on the percussive touchpad. We demonstrate the interest of this new kind of multisensory control via a musical game in which participants collaborate with the machine in order to imitate the sound of an unknown percussive instrument as quickly as possible. Our findings show the challenge and promise of future research in musical "Human–AI parternships".

Domaines

Son [cs.SD]
Fichier principal
Vignette du fichier
jankowiak2024internoise.pdf (1.31 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04579720 , version 1 (18-05-2024)

Identifiants

  • HAL Id : hal-04579720 , version 1

Citer

Loïc Jankowiak, Han Han, Vincent Lostanlen, Mathieu Lagrange. Towards multisensory control of physical modeling synthesis. International Congress & Exposition on Noise Control Engineering (Inter-Noise), Aug 2024, Nantes, France. ⟨hal-04579720⟩
0 Consultations
0 Téléchargements

Partager

Gmail Mastodon Facebook X LinkedIn More