A Multi-FoV Viewport-based Visual Saliency Model Using Adaptive Weighting Losses for 360° Images - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Multimedia Année : 2021

A Multi-FoV Viewport-based Visual Saliency Model Using Adaptive Weighting Losses for 360° Images

Résumé

360° media allows observers to explore the scene in all directions. The consequence is that the human visual attention is guided by not only the perceived area in the viewport but also the overall content in 360°. In this paper, we propose a method to estimate the 360° saliency map which extracts salient features from the entire 360° image in each viewport in three different Field of Views (FoVs). Our model is first pretrained with a large scale 2D image dataset to enable the interpretation of semantic contents, then fine-tuned with a relative small 360° image dataset. A novel weighting loss function attached with stretch weighted maps is introduced to adaptively weight the losses of three evaluation metrics and attenuate the impact of stretched regions in equirectangular projection during training process. Experimental results demonstrate that our model achieves better performance with the integration of three FoVs and its diverse viewport images. Results also show that the adaptive weighting losses and stretch weighted maps effectively enhance the evaluation scores compared to the fixed weighting losses solutions. Comparing to other state of the art models, our method surpasses them on three different datasets and ranks the top using 5 performance evaluation metrics on the Salient360! benchmark set.
Fichier principal
Vignette du fichier
Chao et al-2020-A Multi-FoV Viewport-based Visual Saliency Model Using Adaptive Weighting.pdf (7.18 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02881994 , version 1 (25-08-2020)

Identifiants

Citer

Fang-Yi Chao, Lu Zhang, Wassim Hamidouche, Olivier Déforges. A Multi-FoV Viewport-based Visual Saliency Model Using Adaptive Weighting Losses for 360° Images. IEEE Transactions on Multimedia, 2021, 23, pp.1811-1826. ⟨10.1109/TMM.2020.3003642⟩. ⟨hal-02881994⟩
178 Consultations
137 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More