Machine learning and reproducibility impact of random numbers
Résumé
Reproducibility stands as a pivotal pillar of the scientific method, bolstering confidence in research outcomes. When the findings of a study cannot be consistently reproduced, it gives rise to concerns regarding the validity of the drawn conclusions. This paper delves into an exploration of the Deep Embedded Clustering algorithm. After a first observation of nonreproducibility, we tried to reconstruct the algorithm and we faced repeatability issues, not being able to obtain identical results from run-to-run with the same hardware and exactly the same environment. We have then studied a number of
avenues, which could lead to such problems. We were finally able to achieve run-to-run repeatability on an identical machine, thanks to a particular insight into stochastic parameters and a proper use of hidden pseudorandom number generators. However, we observed differences from one machine to another, indicating that portability is not guaranteed leading to more investigation. Nevertheless, with repeatability on the same machine, we can initiate a reproducibility study.
Domaines
Informatique [cs]Origine | Fichiers produits par l'(les) auteur(s) |
---|