We address the problem of a Network Operator (NO) owning limited resources at the network edge. The NO wishes to enable advanced services, by virtualizing and allocating such resources among multiple tenants, i.e., third-party Service Providers, co-existing at the edge, with different Quality of Service (QoS) constraints. The NO applies a resource allocation policy with the objective of minimizing energy consumption via switching off non-used resources while guaranteeing tenants QoS requirements. We propose a resource allocation policy based on Markov Decision Processes (MDP). In simulation we show that our policy is able to reduce energy consumption, by turning off unused resources, while meeting heterogeneous SP requirements. Our code is available as open source.