Communication Dans Un Congrès Année : 2025

Real-time speech enhancement in noise for throat microphone using neural audio codec as foundation model

Résumé

We present a real-time speech enhancement demo using speech captured with a throat microphone. This demo aims to showcase the complete pipeline, from recording to deep learningbased post-processing, for speech captured in noisy environments with a body-conducted microphone. The throat microphone records skin vibrations, which naturally attenuate external noise, but this robustness comes at the cost of reduced audio bandwidth. To address this challenge, we fine-tune Kyutai's Mimi-a neural audio codec supporting real-time inference-on Vibravox, a dataset containing paired air-conducted and throat microphone recordings. We compare this enhancement strategy against state-of-the-art models and demonstrate its superior performance. The inference runs in an interactive interface that allows users to toggle enhancement, visualize spectrograms, and monitor processing latency.

Fichier principal
Vignette du fichier
WASPAA2025-292.pdf (1.03 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05518258 , version 1 (19-02-2026)

Licence

Identifiants

Citer

Julien Hauret, Thomas Joubaud, Eric Bavu. Real-time speech enhancement in noise for throat microphone using neural audio codec as foundation model. 2025 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), IEEE, Oct 2025, Tahoe City, California, United States. ⟨hal-05518258⟩
74 Consultations
63 Téléchargements

Altmetric

Partager

  • More