Endowing robots with the capability to learn is an important goal for the robotics re-search community. One part of this research is focused on learning skills, where usually two learning paradigms are used sequentially. First, a robot learns a motor primitive by demonstration (or imitation). Then, it improves this motor primitive with respect to some externally defined criterion. In this paper, we study how the representation used in the demonstration learning step can influence the performance of the policy improvement step. We provide a conceptual survey of different demonstration learning al-gorithms and perform an empirical comparison of their performance when combined with a subsequent policy improvement step.