Enabling Markovian Representations under Imperfect Information - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

Enabling Markovian Representations under Imperfect Information

(1, 2) , (2) , (3, 4, 5, 6)
1
2
3
4
5
6

Abstract

Markovian systems are widely used in reinforcement learning (RL), when the successful completion of a task depends exclusively on the last interaction between an autonomous agent and its environment. Unfortunately, real-world instructions are typically complex and often better described as non-Markovian. In this paper we present an extension method that allows solving partially-observable non-Markovian reward decision processes (PONMRDPs) by solving equivalent Markovian models. This potentially facilitates Markovian-based state-of-the-art techniques, including RL, to find optimal behaviours for problems best described as PONMRDP. We provide formal optimality guarantees of our extension methods together with a counterexample illustrating that naive extensions from existing techniques in fully-observable environments cannot provide such guarantees.

Dates and versions

hal-03779034 , version 1 (16-09-2022)

Identifiers

Cite

Francesco Belardinelli, Borja G. León, Vadim Malvone. Enabling Markovian Representations under Imperfect Information. 14th International Conference on Agents and Artificial Intelligence (ICAART 2022), Feb 2022, Online Streaming, France. pp.450-457, ⟨10.5220/0010882200003116⟩. ⟨hal-03779034⟩
6 View
0 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More