A resource allocation problem is a problem in which a set of agents - cooperative or not - must find an assignment of a set of resources. This allocation must match, as best as possible, with the agents’ preferences, which are often antagonist. In most allocation problems, the solution space has a combinatorial structure that creates difficulties with the preference formal representation and the optimal choice calculation. Furthermore, agents must frequently make a decision with an incomplete knowledge of the system state while exogenous factors may affect this state after the decision. There is thus a need for the agents to handle uncertainty about the system in order to maximize their satisfaction. However, the allocation problem gets more complicated in case of uncertainty and usual models of collective resource allocation are not appropriate for this context. Whereas resource allocation models for combinatorial domains are the subject of studies, none of these models consider uncertainty aspects in a distributed context. In this paper, we propose a decision-theoretic approach that allows a set of agents to solve, in a distributed way, resource allocation problems under uncertainty. We represent possible interactions and limited observability as an interaction graph and we propose an MDP based approach to compute exchange strategies taking into account future interaction opportunities.