Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of Mathematical Analysis and Applications Année : 2016

Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures

Résumé

In this paper, we propose an approach for approximating the value function and an ϵ-optimal policy of continuous-time Markov decision processes with Borel state and action spaces, with possibly unbounded cost and transition rates, under the total expected discounted cost optimality criterion. Under adequate assumptions, which in particular include that the transition rate has a density function with respect to a reference measure, together with piecewise Lipschitz continuity of the elements of the control model, we approximate the original controlled process by a model with finite state and action spaces. The approximation error is related to the 1-Wasserstein distance between suitably defined probability measures and approximating measures with finite support. We also study the case when the reference measure is approximated with empirical distributions and we show that convergence of the approximations takes place at an exponential rate in probability.

Dates et versions

hal-01412615 , version 1 (08-12-2016)

Identifiants

Citer

Jonatha Anselmi, François Dufour, Tomás Prieto-Rumeau. Computable approximations for continuous-time Markov decision processes on Borel spaces based on empirical measures. Journal of Mathematical Analysis and Applications, 2016, 443 (2), pp.1323 - 1361. ⟨10.1016/j.jmaa.2016.05.055⟩. ⟨hal-01412615⟩
108 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More