A Hitchhiker's Guide to Statistical Comparisons of Reinforcement Learning Algorithms - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2019

A Hitchhiker's Guide to Statistical Comparisons of Reinforcement Learning Algorithms

Résumé

Consistently checking the statistical significance of experimental results is the first mandatory step towards reproducible science. This paper presents a hitch-hiker's guide to rigorous comparisons of reinforcement learning algorithms. After introducing the concepts of statistical testing, we review the relevant statistical tests and compare them empirically in terms of false positive rate and statistical power as a function of the sample size (number of seeds) and effect size. We further investigate the robustness of these tests to violations of the most common hypotheses (normal distributions, same distributions, equal variances). Beside simulations, we compare empirical distributions obtained by running Soft-Actor Critic and Twin-Delayed Deep Deterministic Policy Gradient on Half-Cheetah. We conclude by providing guidelines and code to perform rigorous comparisons of RL algorithm performances.
Fichier principal
Vignette du fichier
1904.06979.pdf (568.38 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02369859 , version 1 (19-11-2019)

Identifiants

Citer

Cédric Colas, Olivier Sigaud, Pierre-Yves Oudeyer. A Hitchhiker's Guide to Statistical Comparisons of Reinforcement Learning Algorithms. ICLR Worskhop on Reproducibility, May 2019, Nouvelle-Orléans, United States. ⟨hal-02369859⟩
118 Consultations
699 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More