A first approach to an autonomous virtual agent able to play body percussion with real body percussionists is presented. The agent is autonomous in the sense that it can recognize the artists’ calls and react to them by playing back a prerecorded sequence. The agent architecture is described focusing mainly on the artists’ calls recognition module. This work, still under construction, produced two artistic performances which were presented in front of an audience.