This paper investigates the conditions under which cues from social signals can be used for user adaptation (or user tracking) of a learning agent. In this work we consider the case of the Reinforcement Learning (RL) of a dialogue management module. Social signals (gazes, postures, emotions, etc.) have an undeniable importance in human interactions and can be used as an additional and user-dependent (subjective) reinforcement signal during learning. In this paper, the Kalman Temporal Differences (KTD) framework is employed in combination with a potential-based shaping reward method to properly integrate the social information in the optimisa-tion procedure and adapt the policy to user profiles. In a second step the ability of the method to track a new user profile (after self learning of the user or switch to a new user) is shown. Experiments carried out using a state-of-the-art goal-oriented dialogue management framework with simulations support our claims.