Kriging is a widely employed technique, in particular for computer experiments, in machine learning or in geostatistics. An important challenge for Kriging is the computational burden when the data set is large. We focus on a class of methods aiming at decreasing this computational cost, consisting in aggregating Kriging predictors based on smaller data subsets. We prove that aggregations based solely on the conditional variances provided by the different Kriging predictors can yield an inconsistent final Kriging prediction. In contrasts, we study theoretically the recent proposal by [Rullière et al., 2017] and obtain additional attractive properties for it. We prove that this predictor is consistent, we show that it can be interpreted as an exact conditional distribution for a modified process and we provide error bounds for it.