This study pertains to our ongoing work about social artificial vocal interactive agents and their adaptation to users. In this regard, several possibilities to introduce humorous productions in a spoken dialogue system are investigated in order to enhance naturalness during interactions between the agent and the user. Our goal is twofold: automation and optimisation of the humor trait generation process. In this regard, a reinforcement learning scheme is proposed allowing to optimise the usage of humor modules in accordance with user preferences. Some simulated experiments are carried out to confirm that the trained policy used by the humor manager is able to converge to a predefined user profile. Then, some user trials are done to evaluate both the nature of the produced humor and its timely and proportionate usage.