Continuously adjusting the horizontal scaling ofapplications hosted by data centers appears as a good candidateto automatic control approaches allocating resources in closedloopgiven their current workload. Despite several attempts,real applications of these techniques in cloud computing infrastructuresface some difficulties. Some of them essentially turnback to the core concepts of automatic control: controllability,inertia of the controlled system, gain and stability. In thispaper, considering our recent work to build a managementframework dedicated to automatic resource allocation in virtualizedapplications, we attempt to identify from experiments thesources of instabilities in the controlled systems. As examples,we analyze two types of policies: threshold-based and reinforcementlearning techniques to dynamically scale resources. Theexperiments show that both approaches are tricky and thattrying to implement a controller without looking at the waythe controlled system reacts to actions, both in time and inamplitude, is doomed to fail. We discuss both lessons learnedfrom the experiments in terms of simple yet key points to buildgood resource management policies, and longer term issueson which we are currently working to manage contracts andreinforcement learning efficiently in cloud controllers.