Chapitre D'ouvrage Année : 2026

Decisiveness for Countable MDPs and Insights for NPLCSs and POMDPs

Résumé

Markov chains and Markov decision processes (MDPs) are well-established probabilistic models. While finite Markov models are well-understood, analyzing their infinite counterparts remains a significant challenge. Decisiveness has proven to be an elegant property for countable Markov chains: it is general enough to be satisfied by several natural classes of countable Markov chains, and it is a sufficient condition for simple qualitative and approximate quantitative model-checking algorithms to exist. In contrast, existing works on the formal analysis of countable MDPs usually rely on ad hoc techniques tailored to specific classes. We provide here a general framework to analyze countable MDPs by extending the notion of decisiveness. Compared to Markov chains, MDPs exhibit extra non-determinism that can be resolved in an adversarial or cooperative way, leading to multiple natural notions of decisiveness. We show that these notions enable the approximation of reachability and safety probabilities in countable MDPs using simple model-checking procedures. We then instantiate our generic approach to two concrete classes of models inducing countable MDPs: non-deterministic probabilistic lossy channel systems and partially observable MDPs. This leads to an algorithm to approximately compute safety probabilities in each of these classes.

Fichier non déposé

Dates et versions

hal-05344326 , version 1 (03-11-2025)

Identifiants

Citer

Nathalie Bertrand, Patricia Bouyer, Thomas Brihaye, Paulin Fournier, Pierre Vandenhove. Decisiveness for Countable MDPs and Insights for NPLCSs and POMDPs. Principles of Formal Quantitative Analysis, 15760, Springer Nature Switzerland, pp.70-98, 2026, Lecture Notes in Computer Science, 978-3-031-97438-0. ⟨10.1007/978-3-031-97439-7_3⟩. ⟨hal-05344326⟩
63 Consultations
0 Téléchargements

Altmetric

Partager

  • More