Action Decomposable MDP A Linear Programming formulation for queuing network problems
Résumé
Markov decision processes (MDP) have provided general frameworks for many control, decision making, and stochastic optimization problems. In this paper we are interested in a class of queuing control that can be modelled with a Continuous-Time MDP but that have an exponential actions state space with classic methods. The Event Based Dynamic Programming approach deals with this problem and provides some algorithms (value iteration) to compute the best policy efficiently. However there is no formal definition of the subclass of MDP models they can tackle. The first contribution of this paper to define this class, naming it "Action Decomposable MDP". The second contribution is to give a new MDP Linear Programming formulation using "Action Decomposability" that contributes to extend MDP solution techniques. Finally we give some examples of application of this framework and give numeric experiments showing the interest of using the action decomposition properties.
Domaines
Recherche opérationnelle [math.OC]
Origine : Fichiers produits par l'(les) auteur(s)