Optimizing the operation of cooperative multi-agent systems that can deal with large and realistic problems has become an important focal area of research in the multi-agent community. In this paper we first present a new model, the OC-DEC-MDP (Opportunity Cost Decentralized Markov Decision Processes), that allows for representing large multi-agent decision problems with temporal and precedence constraints. Then, we propose polynomial algorithms to efficiently solve problems formalized by OC-DEC-MDPs. The problems we deal with consist of a set of agents that have to execute a set of tasks in a cooperative way. The agents cannot communicate during execution and they have to respect some resource and temporal constraints. Our approach is based on Decentralized Markov Decision Processes (DEC-MDPs) and uses a concept of opportunity cost borrowed from economics to obtain approximate control policies. Currently, the best existing techniques can only solve optimally small problems. Experimental results show that our approach produces good quality solutions for complex problems which are out of reach of existing approaches.