Masking Strategies for Background Bias Removal in Computer Vision Models - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Masking Strategies for Background Bias Removal in Computer Vision Models

Résumé

Models for fine-grained image classification tasks, where the difference between some classes can be extremely subtle and the number of samples per class tends to be low, are particularly prone to picking up background-related biases and demand robust methods to handle potential examples with out-of-distribution (OOD) backgrounds. To gain deeper insights into this critical problem, our research investigates the impact of background-induced bias on finegrained image classification, evaluating standard backbone models such as Convolutional Neural Network (CNN) and Vision Transformers (ViT). We explore two masking strategies to mitigate background-induced bias: Early masking, which removes background information at the (input) image level, and late masking, which selectively masks high-level spatial features corresponding to the background. Extensive experiments assess the behavior of CNN and ViT models under different masking strategies, with a focus on their generalization to OOD backgrounds. The obtained findings demonstrate that both proposed strategies enhance OOD performance compared to the baseline models, with early masking consistently exhibiting the best OOD performance. Notably, a ViT variant employing GAP-Pooled Patch tokenbased classification combined with early masking achieves the highest OOD robustness. Our code and models are available at: https://github.com/ananthu-aniraj/masking_strategies_bias_removal
Fichier principal
Vignette du fichier
ICCV_camera-ready.pdf (1.33 Mo) Télécharger le fichier
ICCVW 2023-00020-supp.pdf (131.01 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04184449 , version 1 (21-08-2023)

Licence

Paternité

Identifiants

  • HAL Id : hal-04184449 , version 1

Citer

Ananthu Aniraj, Cassio F. Dantas, Dino Ienco, Diego Marcos. Masking Strategies for Background Bias Removal in Computer Vision Models. Out of Distribution Generalization in Computer Vision workshop (in conjunction with ICCV 2023), Oct 2023, Paris, France. ⟨hal-04184449⟩
118 Consultations
77 Téléchargements

Partager

Gmail Facebook X LinkedIn More