Real-time learning from data streams in non-stationary environments gains ever more relevance due to the exponentially increasing amounts of generated data. Recently, the Self-Adjusting Memory (SAM) was proposed, an algorithm able to robustly handle heterogeneous types on the basis of two dedicated memories for the current and former concepts that continuously preserve consistency with explicit filtering. Yet, since the algorithm is restricted to one memory architecture, the variety of possible alternatives is limited by design in favor of an overall model consistency. Moreover, it does not actively detect drift, thus adapting with a relatively high delay in case of abrupt changes. We propose a dynamic ensemble on the basis of the SAM algorithm, which is triggered by both, the inherent passive adaptation of SAM and active drift detection. Further, since SAM is based on the stable k-Nearest-Neighbor algorithm, we investigate multiple approaches to obtain a high diversity in the ensemble, resulting in an effective overall strategy. The increased computational demand is countered on the basis of a parallel implementation. We extensively evaluate the method on numerous benchmarks, where it consistently achieves superior results in comparison to state-of-the-art methods.