Low-cost Multispectral Scene Analysis with Modality Distillation - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Low-cost Multispectral Scene Analysis with Modality Distillation

Résumé

Despite its robust performance under various illumination conditions, multispectral scene analysis has not been widely deployed due to two strong practical limitations: 1) thermal cameras, especially high-resolution ones are much more expensive than conventional visible cameras; 2) the most commonly adopted multispectral architectures, twostream neural networks, nearly double the inference time of a regular mono-spectral model which makes them impractical in embedded environments. In this work, we aim to tackle these two limitations by proposing a novel knowledge distillation framework named Modality Distillation (MD). The proposed framework distils the knowledge from a high thermal resolution two-stream network with featurelevel fusion to a low thermal resolution one-stream network with image-level fusion. We show on different multispectral scene analysis benchmarks that our method can effectively allow the use of low-resolution thermal sensors with more compact one-stream networks.
Fichier principal
Vignette du fichier
WACV2022_MD.pdf (7.13 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03491950 , version 1 (18-12-2021)

Identifiants

  • HAL Id : hal-03491950 , version 1

Citer

Heng Zhang, Elisa Fromont, Sébastien Lefèvre, Bruno Avignon. Low-cost Multispectral Scene Analysis with Modality Distillation. WACV 2022 - IEEE Winter Conference on Applications of Computer Vision, Jan 2022, Waikoloa /Virtual, United States. pp.1-10. ⟨hal-03491950⟩
35 Consultations
41 Téléchargements

Partager

Gmail Facebook X LinkedIn More