We propose a new family of bivariate nonnegative integer-autoregressive (BINAR) models for count process data. We first generalize the existing BINAR(1) model by allowing for dependent thinning operators and arbitrary innovation distribution. The extended family allows for intuitive interpretation, as well as tractable aggregation and stationarity properties. We then introduce higher order BINAR(p) and BINAR(∞) dynamics to accommodate more flexible serial dependence patterns. So far, the literature has regarded such models as computationally intractable. We show that the extended BINAR family allows for closed-form predictive distributions at any horizons and for any values of p, which significantly facilitates non-linear forecasting and likelihood based estimation. Finally, a BINAR(∞) model with memory persistence is applied to open-ended mutual fund purchase and redemption order counts