Aggregating optimistic planning trees for solving markov decision processes - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2013

Aggregating optimistic planning trees for solving markov decision processes

Raphael Fonteneau
  • Fonction : Auteur
  • PersonId : 862040
Remi Munos
  • Fonction : Auteur
  • PersonId : 836863

Résumé

This paper addresses the problem of online planning in Markov decision processes using a generative model and under a budget constraint. We propose a new algorithm, ASOP, which is based on the construction of a forest of single successor state planning trees, where each tree corresponds to a random realization of the stochastic environment. The trees are explored using a "safe" optimistic planning strategy which combines the optimistic principle (in order to explore the most promising part of the search space first) and a safety principle (which guarantees a certain amount of uniform exploration). In the decision-making step of the algorithm, the individual trees are aggregated and an immediate action is recommended. We provide a finite-sample analysis and discuss the trade-off between the principles of optimism and safety. We report numerical results on a benchmark problem showing that ASOP performs as well as state-of-the-art optimistic planning algorithms.
Fichier principal
Vignette du fichier
nips13a.pdf (137.36 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-00923681 , version 1 (03-01-2014)

Identifiants

  • HAL Id : hal-00923681 , version 1

Citer

Gunnar Kedenburg, Raphael Fonteneau, Remi Munos. Aggregating optimistic planning trees for solving markov decision processes. Advances in Neural Information Processing Systems, 2013, United States. pp.2382-2390. ⟨hal-00923681⟩
154 Consultations
120 Téléchargements

Partager

Gmail Facebook X LinkedIn More