Multimodal chaptering for long-form TV newscast video - Archive ouverte HAL
Pré-Publication, Document De Travail Année : 2024

Multimodal chaptering for long-form TV newscast video

Résumé

We propose a novel approach for automatic chaptering of TV newscast videos, addressing the challenge of structuring and organizing large collections of unsegmented broadcast content. Our method integrates both audio and visual cues through a two-stage process involving frozen neural networks and a trained LSTM network. The first stage extracts essential features from separate modalities, while the LSTM effectively fuses these features to generate accurate segment boundaries. Our proposed model has been evaluated on a diverse dataset comprising over 500 TV newscast videos of an average of 41 minutes gathered from TF1, a French TV channel, with varying lengths and topics. Experimental results demonstrate that this innovative fusion strategy achieves state of the art performance, yielding a high precision rate of 82% at IoU of 90%. Consequently, this approach significantly enhances analysis, indexing and storage capabilities for TV newscast archives, paving the way towards efficient management and utilization of vast audiovisual resources.
Fichier principal
Vignette du fichier
paper_newscast_chaptering_arxiv.pdf (1.2 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04511269 , version 1 (19-03-2024)

Licence

Identifiants

  • HAL Id : hal-04511269 , version 1

Citer

Khalil Guetari, Yannis Tevissen, Frédéric Petitpont. Multimodal chaptering for long-form TV newscast video. 2024. ⟨hal-04511269⟩
87 Consultations
106 Téléchargements

Partager

More