Distributionally robust chance-constrained Markov decision processes with random transition probabilities - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

Distributionally robust chance-constrained Markov decision processes with random transition probabilities

Hoang Nam Nguyen
  • Fonction : Auteur
  • PersonId : 1110006
Abdel Lisser
Vikas Vikram Singh
  • Fonction : Auteur
  • PersonId : 1088211

Résumé

Markov decision process (MDP) is a decision making framework where a decision maker is interested in maximizing the expected discounted value of a stream of rewards received at future stages at various states which are visited according to a controlled Markov chain. Many algorithms including linear programming methods are available in the literature to compute an optimal policy when the rewards and transition probabilities are deterministic. In this paper, we consider an MDP problem where the reward vector is known and the transition probability vector is a random vector which follow a discrete distribution whose information is not completely known. We formulate the MDP problem using distributionally robust chance-constrained optimization framework under various types of moments based uncertainty sets, and statistical-distance based uncertainty sets defined using ϕ-divergence and Wasserstein distance metric. For each uncertainty set, we propose an equivalent mix-integer bilinear programming problem or a mix-integer semidefinite programming problem with bilinear constraints. As an application, we study a machine replacement problem and perform numerical experiments on randomly generated instances.
Fichier principal
Vignette du fichier
DRO_MDP_Random_transition_probabilities.pdf (585.16 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04373180 , version 1 (04-01-2024)

Identifiants

  • HAL Id : hal-04373180 , version 1

Citer

Hoang Nam Nguyen, Abdel Lisser, Vikas Vikram Singh. Distributionally robust chance-constrained Markov decision processes with random transition probabilities. 2024. ⟨hal-04373180⟩
44 Consultations
45 Téléchargements

Partager

Gmail Facebook X LinkedIn More