Ensemble methods combining several models have shown superior predictive performance in data streams forecasting compared to individual models. Besides, they can cope with evolving data streams and concept drift as they allow adaptation. However, ensembles are renowned for their complexity and computational costs which makes them unsuitable in cases where both resources and time are limited such as IoT applications. In this paper, we propose to use model compression in the streaming setting in order to overcome the aforementioned drawbacks. We show that compressing a highly performing dynamic ensemble into an individual model leads to better predictive performance when compared to an individual learner while significantly reducing computational costs. We conduct an extensive experimental study on both real and synthetic time series to measure the impact of compression on both predictive performance and computational cost.