This paper proposes a methodology for the semi-automated design of a game AI for simulation and strategy games which require the player to control a potentially high number of characters or units in complex environments. After defending the idea of using Machine Learning, and especially Reinforcement Learning, as a basic technique, some of the key issues, mostly dealing with complexity, representation, and coordination, are outlined, and some ways forward are proposed. These revolve mainly around the idea of problem decomposition using the game structure, and of bootstrapping the learning process by letting the learning game AI play against another AI, and only progressively take more and more control of the decision-making. The ongoing application of this methodology to the design of a new game AI for an existing wargame system is described in some detail.