In this paper, an approach is presented to automatically allocate a set of exploration tasks between a fleet of mobile robots. The approach combines a Road-Map technique and Markovian Decision Processes (MDPs). The addressed problem consists of exploring an area where a set of points of interest characterizes the main positions to be visited by the robots. This problem induces a long term horizon motion planning with a combinatorial explosion. The Road-Map allows the robots to represent their spatial knowledge as a graph of way-points connected by paths. It can be modified during the exploration mission requiring the robots to use on-line computations. By decomposing the Road-Map into regions, an MDP allows the current group leader to evaluate the interest of each robot in every single region. Using those values, the leader can assign the exploration tasks to the robots.