The development of tool use in children is a key question for the understanding of the development of human cognition. Several authors have studied it to investigate how children explore, evaluate and select alternative strategies for solving problems. In particular, Siegler has used this domain to develop the overlapping waves theory that characterizes how infants continue to explore alternative strategies to solve a par- ticular problem, even when one is currently better than others. In computational models of strategy selection for the problem of integer addition, Shrager and Siegler proposed a mechanism that maintains the concurrent exploration of alternative strategies with use frequencies that are proportional to their performance for solving a particular problem. This mechanism was also used by Chen and Siegler to interpret an experiment with 1.5- and 2.5-year-olds that had to retrieve an out-of-reach toy, and where they could use one of several available strategies that included leaning forward to grasp a toy with the hand or using a tool to retrieve the toy. In this paper, we use this domain of tool use discovery to consider other mechanisms of strategy selection and evaluation. In particular, we present models of curiosity- driven exploration, where strategies are explored according to the learning progress/information gain they provide (as opposed to their current efficiency to actually solve the problem). In these models, we define a curiosity-driven agent learning a hierarchy of different sensorimotor models in a simple 2D setup with a robotic arm, a stick and a toy. In a first phase, the agent learns from scratch how to use its robotic arm to control the tool and to catch the toy, and in a second phase with the same learning mechanisms, the agent has to solve three problems where the toy can only be reached with the tool. We show that agents choosing strategies based on a learning progress measure also display overlapping waves of behavior compatible with the one observed in infants, and we suggest that curiosity-driven exploration could be at play in Chen and Siegler’s experiment, and more generally in tool use discovery.