In this paper, we use a gene regulatory network (GRN) to regulate a reinforcement learning controller, the State- Action-Reward-State-Action (SARSA) algorithm. The GRN serves as a neuromodulator of SARSA's learning parame- ters: learning rate, discount factor, and memory depth. We have optimized GRNs with an evolutionary algorithm to regulate these parameters on specific problems but with no knowledge of problem structure. We show that genetically- regulated neuromodulation (GRNM) performs comparably or better than SARSA with fixed parameters. We then ex- tend the GRNM SARSA algorithm to multi-task problem generalization, and show that GRNs optimized on multi- ple problem domains can generalize to previously unknown problems with no further optimization.