Reinforcement-learning for sampling design in Markov random fields - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2012

Reinforcement-learning for sampling design in Markov random fields

Résumé

Optimal sampling in spatial random fields is a complex problem, which mobilizes several research fields in spatial statistics and artificial intelligence. We consider the case where observations are discrete-valued and modelled by a Markov Random Field. Then we encode the sampling problem into the Markov Decision Process (MDP) framework. After exploring existing heuristic solutions as well as classical algorithms from the field of Reinforcement Learning (RL), we design an original algorithm, LSDP (Least Square Dynamic Programming), which uses simulated trajectories to solve approximately any finite-horizon MDP problem. Based on an empirical study of the behaviour of these different approaches on binary models, we derive the following conclusions: i) a naive heuristic, consisting in sampling sites where marginals are the most uncertain, is already an efficient sampling approach. ii) LSDP outperforms all the classical RL approaches we have tested. iii) LSDP outperforms the heuristic in cases when reconstruction errors have a high cost, or sampling actions are constrained. In addition, LSDP readily handles action costs in the optimisation problem, as well as cases when some sites of the MRF can not be observed.
Fichier principal
Vignette du fichier
BPS_CompStat2012_{4243E05E-7AE8-4E6E-A43A-B908BD6DC9F7}.pdf (341.21 Ko) Télécharger le fichier
Origine : Accord explicite pour ce dépôt
Loading...

Dates et versions

hal-01191361 , version 1 (01-09-2015)

Identifiants

  • HAL Id : hal-01191361 , version 1
  • PRODINRA : 179451

Citer

Mathieu Bonneau, Nathalie Dubois Peyrard, Régis Sabbadin. Reinforcement-learning for sampling design in Markov random fields. International Conference on Computational Statistics, Aug 2012, Limassol, Cyprus. pp.12. ⟨hal-01191361⟩
68 Consultations
155 Téléchargements

Partager

Gmail Facebook X LinkedIn More