Bi-objective Lexicographic Optimization in Markov Decision Processes with Related Objectives - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Bi-objective Lexicographic Optimization in Markov Decision Processes with Related Objectives

Debraj Chakraborty
Anirban Majumdar
Sayan Mukherjee
Guillermo Pérez
Jean-François Raskin

Résumé

We consider lexicographic bi-objective problems on Markov Decision Processes (MDPs), where we optimize one objective while guaranteeing optimality of another. We propose a two-stage technique for solving such problems when the objectives are related (in a way that we formalize). We instantiate our technique for two natural pairs of objectives: minimizing the (conditional) expected number of steps to a target while guaranteeing the optimal probability of reaching it; and maximizing the (conditional) expected average reward while guaranteeing an optimal probability of staying safe (w.r.t. some safe set of states). For the first combination of objectives, which covers the classical frozen lake environment from reinforcement learning, we also report on experiments performed using a prototype implementation of our algorithm and compare it with what can be obtained from state-of-the-art probabilistic model checkers solving optimal reachability.

Dates et versions

hal-04442545 , version 1 (06-02-2024)

Identifiants

Citer

Damien Busatto-Gaston, Debraj Chakraborty, Anirban Majumdar, Sayan Mukherjee, Guillermo Pérez, et al.. Bi-objective Lexicographic Optimization in Markov Decision Processes with Related Objectives. 21st International Symposium on Automated Technology for Verification and Analysis (ATVA 2023), Oct 2023, Singapore, Singapore. pp.203-223, ⟨10.1007/978-3-031-45329-8_10⟩. ⟨hal-04442545⟩

Collections

LACL UPEC
6 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More