Inter-individual differences in infants' numerosity processing have been assessed using a change detection paradigm, where participants were presented with two concurrent streams of images, one alternating between two numerosities and the other showing one constant numerosity. While most infants look longer at the changing stream in this paradigm, the reasons underlying these preferences have remained unclear. We suggest that, besides being attracted by numerosity changes, infants perhaps also respond to the alternating pattern of the changing stream. We conducted two experiments (N = 32) with 6-month-old infants to assess this hypothesis. In the first experiment, infants responded to changes in numerosity even when the changing stream showed numerosities in an unpredictable random order. In the second experiment, infants did not display any preference when an alternating stream was pitted against a random stream. These findings do not provide evidence that the alternating pattern of the changing stream contributes to drive infants' preferences. Instead, around the age of 6 months, infants' responses in the numerosity change detection paradigm appear to be mainly driven by changes in numerosity, with different levels of preference reflecting inter-individual difference in the acuity of numerosity perception.