In this paper, we describe FACS, a new Michigan style architecture able to build Finite-State Automata controllers for agents learning to solve non-Markov problems. FACS relies on a population of partial automata and implements a Reinforcement Learning algorithm to compute the strength of each automaton and a Genetic Algorithm to select and discover efficient automata. We detail our approach and present very preliminary results.