In some African protected areas, elephant populations have reached high densities causing concern about their influence on other forms of biodiversity, and in particular, other large herbivores. This has led to a debate whether management of elephant populations might need to be implemented. Surprisingly, few studies have focused on the influence of elephants on other herbivores and the existing results are conflicting. We investigated such influence in Hwange National Park, Zimbabwe, where elephants were culled and maintained at low density up to 1986, and have more than doubled since then, thus providing the opportunity to explore their possible influence on other herbivore species. Trends of different herbivore populations were obtained using long-term waterhole census data, with these results also being correlated to changes in dry season rainfall, as another fundamental determinant of herbivore population fluctuations. While our study did not allow us to properly disentangle the role of elephants and rainfall, it provided valuable insights. For five out of the 13 herbivore species studied, the population increased between 1973 and 1986 (high dry season rainfall; relatively low elephant abundance), then decreased between 1987 and 1995 (extremely low dry season rainfall; increase in the elephant population), and finally have increased since 1996 (high dry season rainfall; high elephant abundance). These results are consistent with an important role of dry season rainfall on herbivore population dynamics. Additionally, for four other species, the population increased between 1973 and 1986 and has decreased since then. As most herbivore declines occurred simultaneously with increase in the elephant population in the 1987–1995 period, and with the recent increase in most populations happening at a lower rate than the 1973–1985 period, our results suggest the possible existence of a negative influence of high elephant densities on other herbivores. We discuss what the possible underlying mechanisms can be and how to investigate them.