Wargames are an example of complex multiagent simulations for which, specifying agent behavior adequately in advance for all potential situations is not feasible. In this context, we have applied reinforcement learning as an adaptive approach to design strategies for these simulations. In this paper, we introduce our approach and focus on a novel algorithm for generating representations with adequate granularities for commanders of a military hierarchy.