In this work we develop an online deep-learning based approach for classification on data streams. Our approach is able to learn in an incremental way without storing and reusing the historical data (we only store a recent history) while processing each new data sample only once. To make up for the absence of the historical data, we train Generative Adversarial Networks (GANs), which, in recent years have shown their excellent capacity to learn data distributions for image datasets. We test our approach on MNIST and LSUN datasets and demonstrate its ability to adapt to previously unseen data classes or new instances of previously seen classes, while avoiding forgetting of previously learned classes/instances of classes that do not appear anymore in the data stream.