Bayes-Adaptive Impulse Control of Piecewise-Deterministic Markov Processes
Résumé
We introduce a Bayes-adaptive framework for controlling Piecewise-Deterministic Markov Processes (PDMPs) under partial observability and uncertain parameters. PDMPs form a class of continuous-time Markov processes that capture hybrid (discrete–continuous) dynamics, allowing highly flexible modeling with a small number of interpretable parameters. We cast the problem as a hybrid state-space Bayes-Adaptive Partially Observable Markov Decision Process (BAPOMDP), which accounts for model uncertainty offline, without requiring prior interaction with the system. This BAPOMDP can be seen as a higher-dimensional hybrid Partially Observable Markov Decision Process (POMDP). Because computing optimal policies in such hybrid state-space POMDPs is intractable, we rely on simulation-based deep reinforcement learning algorithms to obtain effective solutions. The approach is demonstrated in a medical patient follow-up scenario, where numerical experiments highlight the feasibility of applying this framework in realistic settings with partial observability and uncertain dynamics.
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |