Batch Inverse-Variance Weighting: Deep Heteroscedastic Regression - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Batch Inverse-Variance Weighting: Deep Heteroscedastic Regression

Résumé

Heteroscedastic regression is the task of supervised learning where each label is subject to noise from a different distribution. This noise can be caused by the labelling process, and impacts negatively the performance of the learning algorithm as it violates the i.i.d. assumptions. In many situations however, the labelling process is able to estimate the variance of such distribution for each label, which can be used as an additional information to mitigate this impact. We adapt an inverse-variance weighted mean square error, based on the Gauss-Markov theorem, for parameter optimization on neural networks. We introduce Batch Inverse-Variance, a loss function which is robust to near-ground truth samples, and allows to control the effective learning rate. Our experimental results show that BIV improves significantly the performance of the networks on two noisy datasets, compared to L2 loss, inverse-variance weighting, as well as a filtering-based baseline.

Dates et versions

hal-04255221 , version 1 (23-10-2023)

Identifiants

Citer

Vincent Mai, Waleed Khamies, Liam Paull. Batch Inverse-Variance Weighting: Deep Heteroscedastic Regression. ICML 2021 Workshop on Uncertainty & Robustness in Deep Learning, Jul 2021, Virtual, Unknown Region. ⟨10.48550/arXiv.2107.04497⟩. ⟨hal-04255221⟩
34 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More