Adaptation is a key mechanism in human-human interaction. In our work, we aim at endowing embodied conversational agents with the ability to adapt their behaviour when interacting with a human interlocutor. With the goal to better understand what are the main challenges concerning adaptive agents, we investigated the effects on user's experience of three adaptation models for a virtual agent. The adaptation mechanisms performed by the agent take into account user's reaction and learn how to adapt on the fly during the interaction. Agent's adaptation is realised at several levels (i.e., at behavioural, conversational and signal level) and focuses on improving user's experience along different dimensions (i.e., user's impressions and engagement). In our first two studies, we aim to learn agent's multi-modal behaviours and conversational strategies to optimise dynamically user's engagement and impressions of the agent, by taking them as input during the learning process. In our third study, our model takes as input both the user's and the agent's past behaviour and predicts the agent's next behaviour. Our adaptation models have been evaluated through experimental studies sharing the same interacting scenario, with the agent playing the role of a virtual museum guide. These studies showed an impact of the adaptation mechanisms on user's experience of the interaction and their perception of the agent. Interacting with an adaptive agent vs a non-adaptive agent tended to be more positively perceived. Finally, the effects of people's a-priori about virtual agents found in our studies highlight the importance to take into account user's expectancies in human-agent interaction.