Hierarchical Framework for Plot De-interlacing of TV Series based on Speakers, Dialogues and Images - Algorithmes Parallèles et Optimisation Accéder directement au contenu
Communication Dans Un Congrès Année : 2012

Hierarchical Framework for Plot De-interlacing of TV Series based on Speakers, Dialogues and Images

Résumé

Since the 90s, TV series tend to introduce more and more main characters and they are often composed of multiple intertwined stories. In this paper, we propose a hierarchical framework of plot de-interlacing which permits to cluster semantic scenes into stories: a story is a group of scenes not necessarily contiguous but showing a strong semantic relation. Each scene is described using three different modalities (based on color histograms, speaker diarization or automatic speech recognition outputs) as well as their multimodal combination. We introduce the notion of character-driven episodes as episodes where stories are emphasized by the presence or absence of characters, and we propose an automatic method, based on a social graph, to detect these episodes. Depending on whether an episode is character-driven or not, the plot-de-interlacing -which is a scene clustering- is made either through a traditional average-link agglomerative clustering with speaker modality only, either through a spectral clustering with the fusion of all modalities. Experiments, conducted on twenty three episodes from three quite different TV series (different lengths and formats), show that the hierarchical framework brings an improvement for all the series.
Fichier non déposé

Dates et versions

hal-01987815 , version 1 (21-01-2019)

Identifiants

  • HAL Id : hal-01987815 , version 1

Citer

Philippe Ercolessi, Christine Sénac, Sandrine Mouysset, Hervé Bredin. Hierarchical Framework for Plot De-interlacing of TV Series based on Speakers, Dialogues and Images. AMVA 2012, 1st ACM International Workshop on Audio and Multimedia Methods for Large-Scale Video Analysis at ACM Multimedia 2012, 2012, Nara, Japan. ⟨hal-01987815⟩
32 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More