Pré-Publication, Document De Travail Année : 2025

Bayes-Adaptive Impulse Control of Piecewise-Deterministic Markov Processes

Résumé

In this paper, we propose a Bayes-adaptive approach for controlling Piecewise-Deterministic Markov Processes (PDMPs) under partial observability and incomplete model knowledge. PDMPs are a form of continuous time (semi-)Markov processes which enable the description of hybrid (discrete-continuous) process dynamics. They allow for the modeling of very general dynamics with a minimal set of interpretable parameters. When some parameters of the controlled PDMP are poorly known, we demonstrate that a Bayes-adaptive approach can provide a learning while managing control method. To achieve this, we show how to formulate the problem as a Bayes-Adaptive Partially Observable Markov Decision Process (BAPOMDP), which itself can be modeled as a higherdimensional Partially Observable Markov Decision Process (POMDP). Subsequently, deep reinforcement learning algorithms can be employed to solve the resulting problem off-line. We illustrate the various steps of our approach with a medical patient follow-up application and utilize the Proximal Policy Optimization (PPO) algorithm to solve the final POMDP model.

Fichier principal
Vignette du fichier
deep_bamcp_po_conf-14.pdf (513.66 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05054392 , version 1 (05-05-2025)
hal-05054392 , version 2 (22-10-2025)
hal-05054392 , version 3 (12-01-2026)
hal-05054392 , version 4 (18-02-2026)

Licence

Identifiants

  • HAL Id : hal-05054392 , version 1

Citer

Orlane Rossini, Meritxell Vinyals, Alice Cleynen, Benoîte de Saporta, Régis Sabbadin. Bayes-Adaptive Impulse Control of Piecewise-Deterministic Markov Processes. 2025. ⟨hal-05054392v1⟩
494 Consultations
409 Téléchargements

Partager

  • More