Communication Dans Un Congrès Année : 2026

Lina-Speech: Gated Linear Attention and Initial-State Tuning for Multi-Sample Prompting Text-To-Speech Synthesis

Résumé

Neural codec language models, built on transformer architecture, have revolutionized textto-speech (TTS) synthesis, excelling in voice cloning by treating it as a prefix continuation task. However, their limited context length hinders their effectiveness to short speech samples. As a result, the voice cloning ability is restricted to a limited coverage and diversity of the speaker's prosody and style. Besides, adapting prosody, accent, or appropriate emotion from a short prefix remains a challenging task. Finally, the quadratic complexity of self-attention limits inference throughput. In this work, we introduce Lina-Speech, a TTS model with Gated Linear Attention (GLA) to replace standard self-attention as a principled backbone, improving inference throughput while matching state-of-the-art performance. Leveraging the stateful property of recurrent architecture, we introduce an Initial-State Tuning (IST) strategy that unlocks the possibility of multiple speech sample conditioning of arbitrary numbers and lengths and provides a comprehensive and efficient strategy for voice cloning and out-of-domain speaking style and emotion adaptation. We demonstrate the effectiveness of this approach for controlling fine-grained characteristics such as prosody and emotion. Code, checkpoints, and demo are freely available: https://github.com/theodorblackbird/lina-speech

Fichier principal
Vignette du fichier
Lina_Speech__Gated_Linear_Attention_and_Initial_State_Tuning_for_Multi_Sample_Prompting_Text_To_Speech_Synthesis.pdf (494.04 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05525317 , version 1 (24-02-2026)

Licence

Identifiants

  • HAL Id : hal-05525317 , version 1

Citer

Théodor Lemerle, Téo Guichoux, Axel Roebel, Nicolas Obin. Lina-Speech: Gated Linear Attention and Initial-State Tuning for Multi-Sample Prompting Text-To-Speech Synthesis. The 40th AAAI Conference on Artificial Intelligence -- Audio-Workshop, Jan 2026, Singapore, Singapore. ⟨hal-05525317⟩
347 Consultations
142 Téléchargements

Partager

  • More