Pré-Publication, Document De Travail Année : 2026

Bayes-Adaptive Impulse Control of Piecewise-Deterministic Markov Processes

Résumé

We introduce a Bayes-adaptive framework for controlling Piecewise-Deterministic Markov Processes (PDMPs) under partial observability and uncertain parameters. PDMPs form a class of continuous-time Markov processes that capture hybrid (discrete–continuous) dynamics, allowing highly flexible modeling with a small number of interpretable parameters. We cast the problem as a hybrid state-space Bayes-Adaptive Partially Observable Markov Decision Process (BAPOMDP), which accounts for model uncertainty offline, without requiring prior interaction with the system. This BAPOMDP can be seen as a higher-dimensional hybrid Partially Observable Markov Decision Process (POMDP). Because computing optimal policies in such hybrid state-space POMDPs is intractable, we rely on simulation-based deep reinforcement learning algorithms to obtain effective solutions. The approach is demonstrated in a medical patient follow-up scenario, where numerical experiments highlight the feasibility of applying this framework in realistic settings with partial observability and uncertain dynamics.

Fichier principal
Vignette du fichier
deep_bamcp_po_conf-31.pdf (445.64 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05054392 , version 1 (05-05-2025)
hal-05054392 , version 2 (22-10-2025)
hal-05054392 , version 3 (12-01-2026)
hal-05054392 , version 4 (18-02-2026)

Licence

Identifiants

  • HAL Id : hal-05054392 , version 3

Citer

Orlane Rossini, Meritxell Vinyals, Alice Cleynen, Benoîte de Saporta, Régis Sabbadin. Bayes-Adaptive Impulse Control of Piecewise-Deterministic Markov Processes. 2026. ⟨hal-05054392v3⟩
494 Consultations
409 Téléchargements

Partager

  • More