Chronic infections of the human immunodeficiency virus (HIV) create a very complex co-evolutionary process, where the virus tries to escape the continuously adapting host immune system. Quantitative details of this process are largely unknown and could help in disease treatment and vaccine development. Here we study a longitudinal dataset of ten HIV-infected people, where both the B-cell receptors and the virus are deeply sequenced. We focus on simple measures of turnover, which quantify how much the composition of the viral strains and the immune repertoire change between time points. At the single-patient level, the viral-host turnover rates do not show any statistically significant correlation, however they correlate if the information is aggregated across patients. In particular, we identify an anti-correlation: large changes in the viral pool composition come with small changes in the B-cell receptor repertoire. This result seems to contradict the naive expectation that when the virus mutates quickly, the immune repertoire needs to change to keep up. However, we show that the observed anti-correlation naturally emerges and can be understood in terms of simple population-genetics models.