Dynamic allocation optimization in A/B tests using classification-based preprocessing - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Knowledge and Data Engineering Année : 2021

Dynamic allocation optimization in A/B tests using classification-based preprocessing

Résumé

An A/B test evaluates the impact of a new technology by running it in a real production environment and testing its performance on a set of items. Recently, promising new methods are optimizing A/B tests with dynamic allocation. They allow for a quicker result regarding which variation (A or B) is the best, saving money for the user. However, dynamic allocation by traditional methods requires certain assumptions, which are not always verified in reality. This is mainly due to the fact that the populations tested are not homogeneous. This document reports on the new reinforcement learning methodology which has been deployed by the commercial A/B testing platform AB Tasty. We provide a new method that not only builds homogeneous groups for a user, but also allows to find the best variation for these groups in a short period of time. This paper provides numerical results on AB Tasty data, but also on public data sets, to demonstrate an improvement in A/B testing over traditional methods.
Fichier principal
Vignette du fichier
TKDE_CTREEUCB_hal.pdf (2.99 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-01874969 , version 1 (15-09-2018)
hal-01874969 , version 2 (23-09-2019)
hal-01874969 , version 3 (13-05-2020)
hal-01874969 , version 4 (23-08-2021)

Licence

Paternité - Pas d'utilisation commerciale

Identifiants

Citer

Emmanuelle Claeys, Pierre Gancarski, Myriam Maumy-Bertrand, Hubert Wassner. Dynamic allocation optimization in A/B tests using classification-based preprocessing. IEEE Transactions on Knowledge and Data Engineering, In press, ⟨10.1109/TKDE.2021.3076025⟩. ⟨hal-01874969v4⟩
284 Consultations
617 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More