SlateFree: a Model-Free Decomposition for Reinforcement Learning with Slate Actions - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2022

SlateFree: a Model-Free Decomposition for Reinforcement Learning with Slate Actions

Résumé

We consider the problem of sequential recommendations, where at each step an agent proposes some slate of N distinct items to a user from a much larger catalog of size K >> N. The user has unknown preferences towards the recommendations and the agent takes sequential actions that optimise (in our case minimise) some user-related cost, with the help of Reinforcement Learning. The possible item combinations for a slate is N-over-K, an enormous number rendering value iteration methods intractable. We prove that the slate-MDP can actually be decomposed using just K item-related Q functions per state, which describe the problem in a more compact and efficient way. Based on this, we propose a novel model-free SARSA and Q-learning algorithm that performs N parallel iterations per step, without any prior user knowledge. We call this method SlateFree, i.e. free-ofslates, and we show numerically that it converges very fast to the exact optimum for arbitrary user profiles, and that it outperforms alternatives from the literature.
Fichier principal
Vignette du fichier
slatefree_2022.pdf (568.29 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03769194 , version 1 (05-09-2022)

Licence

Paternité - Pas d'utilisation commerciale

Identifiants

  • HAL Id : hal-03769194 , version 1

Citer

Anastasios Giovanidis. SlateFree: a Model-Free Decomposition for Reinforcement Learning with Slate Actions. 2022. ⟨hal-03769194⟩
57 Consultations
28 Téléchargements

Partager

Gmail Facebook X LinkedIn More