A Dual Rig Approach for Multi-View Video and Spatialized Audio Capture in Medical Training - Archive ouverte HAL
Communication Dans Un Congrès Année : 2024

A Dual Rig Approach for Multi-View Video and Spatialized Audio Capture in Medical Training

Joshua Maraval
  • Fonction : Auteur
  • PersonId : 1290830
Bangning Wei
  • Fonction : Auteur
  • PersonId : 1413018
David Pesce
  • Fonction : Auteur
  • PersonId : 1413019
Yann Gayral
  • Fonction : Auteur
  • PersonId : 1413020
Nicolas Ramin
  • Fonction : Auteur
  • PersonId : 1290831
Lu Zhang
  • Fonction : Auteur
  • PersonId : 1413021

Résumé

We present a multi-view camera and spatialized audio microphone capture system designed for computer vision applications in free navigation immersive experiences. We propose a dataset of two long and complex in-situ training situations in the medical field. The scenarios in the dataset feature precise gestures for the learner to reproduce during complex situations with multiple simultaneous visual and auditory cues important for training. 3D computer vision techniques are used to reconstruct a 4D scene model from a set of videos to render novel views from unseen viewpoints. However, the quality of the rendered objects is directly dependent on the density of coverage by reference views. To ensure maximum QoE, we propose a dual rig of cameras, a central rig that captures the details of the gesture zone of the training scenarios and a peripheral rig that captures the environment of the room and the interactions occurring around the gesture zone. The central rig provides dense coverage of the central content, facilitating high-quality reconstruction on novel views of the captured gestures. Recordings include audio interactions of multiple actors, captured by ambisonic microphones spatially distributed around the scene. The captured scenes are real-world educational content for medical courses, so this dataset provides a rare opportunity to assess the QoE of volumetric video techniques on realistic content, and to compare their pedagogical capabilities with standard multi-view video content.

Fichier principal
Vignette du fichier
QOMEX2024.pdf (4.75 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04689748 , version 1 (15-11-2024)

Identifiants

Citer

Joshua Maraval, Bangning Wei, David Pesce, Yann Gayral, Meriem Outtas, et al.. A Dual Rig Approach for Multi-View Video and Spatialized Audio Capture in Medical Training. 2024 16th International Conference on Quality of Multimedia Experience (QoMEX), Jun 2024, Karlshamn, France. pp.274-277, ⟨10.1109/QoMEX61742.2024.10598273⟩. ⟨hal-04689748⟩
26 Consultations
10 Téléchargements

Altmetric

Partager

More