Communication Dans Un Congrès Année : 2025

AVS-Net: Audio-Visual Scale Net for Self-supervised Monocular Metric Depth Estimation

Sascha Hornauer
Fabien Moutarde
Jialiang Lu
  • Fonction : Auteur
  • PersonId : 1465535

Résumé

Metric depth prediction from monocular videos suffers from bad generalization between datasets and requires supervised depth data for scale-correct training. Self-supervised training using multi-view reconstruction can benefit from large scale natural videos but not provide correct scale, limiting its benefits. Recently, reflecting audible Echoes off objects is investigated for improved depth prediction and was shown to be sufficient to reconstruct objects at scale even without a visual signal. Because Echoes travel at fixed speed, they have the potential to resolve ambiguities in object scale and appearance. However, predicting depth end-to-end from sound and vision cannot benefit from unsupervised depth prediction approaches, which can process large scale data without sound annotation. In this work we show how Echoes can benefit depth prediction in two ways: When learning metric depth learned from supervised data and as supervisory signal for scale-correct self-supervised training. We show how we can improve the predictions of several state-of-the-art approaches and how the method can scale-correct a self-supervised depth approach.

Mots clés

Fichier principal
Vignette du fichier
Xiaohu_LIU_AVS-Net_Audio-Visual_Scale_Net_for_Self-supervised_Monocular_Metric_Depth_Estimation.pdf (1.63 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05543245 , version 1 (09-03-2026)

Licence

Identifiants

  • HAL Id : hal-05543245 , version 1

Citer

Xiaohu Liu, Sascha Hornauer, Fabien Moutarde, Jialiang Lu. AVS-Net: Audio-Visual Scale Net for Self-supervised Monocular Metric Depth Estimation. Sight and Sound Workshop - IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2025, Andrew Owens, Jun 2025, Nashville, Tennessee, USA, United States. ⟨hal-05543245⟩
35 Consultations
58 Téléchargements

Partager

  • More