Distributionally robust chance-constrained Markov decision processes with random payoff - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

Distributionally robust chance-constrained Markov decision processes with random payoff

Hoang Nam Nguyen
  • Fonction : Auteur
  • PersonId : 1110006
Abdel Lisser
Vikas Vikram Singh
  • Fonction : Auteur
  • PersonId : 1088211

Résumé

A Markov Decision Process (MDP) is a framework used for decision-making. In an MDP problem, the decision maker's goal is to maximize the expected discounted value of future rewards while navigating through different states controlled by a Markov chain. In this paper, we focus on the case where the transition probabilities vector is deterministic, while the reward vector is uncertain and follow a partially known distribution. We employ a distributionally robust chance constraints approach to model the MDP. This approach entails the construction of potential distributions of reward vector, characterized by moments or statistical metrics. We explore two situations for these uncertainty sets: one where the reward vector has a real support and another where it is constrained to be nonnegative. In the case of a real support, we demonstrate that solving the distributionally robust chance-constrained Markov decision process is mathematically equivalent to a second-order cone programming problem for moments and ϕ-divergence uncertainty sets. For Wasserstein distance uncertainty sets, it becomes a mixed-integer second-order cone programming problem. In contrast, when dealing with nonnegative reward vector, the equivalent optimization problems are different. Moments-based uncertainty sets lead to a copositive optimization problem, while Wasserstein distance-based uncertainty sets result in a biconvex optimization problem. To illustrate the practical application of these 1 methods, we examine a machine replacement problem and present results conducted on randomly generated instances to showcase the effectiveness of our proposed methods.
Fichier principal
Vignette du fichier
DRO_MDP_Random_payoff__Applied_Maths_and_Optimization_template_.pdf (612.77 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04373178 , version 1 (04-01-2024)

Identifiants

  • HAL Id : hal-04373178 , version 1

Citer

Hoang Nam Nguyen, Abdel Lisser, Vikas Vikram Singh. Distributionally robust chance-constrained Markov decision processes with random payoff. 2024. ⟨hal-04373178⟩
39 Consultations
59 Téléchargements

Partager

Gmail Facebook X LinkedIn More