In recent years, convolutional neural networks (CNNs) have been widely used in various computer visual recognition tasks and then extensively applied for medical images, particularly for computer-aided diagnosis. Despite this success, CNNs suffer from a catastrophic forgetting issue, making it difficult for them to learn new tasks without being retrained from scratch. In this chapter the authors briefly introduce the concept of incremental learning for deep CNNs and highlight the catastrophic forgetting challenge by discussing methods from the literature that are used to mitigate it, particularly in the field of bioinformatics. To the best of the authors’ knowledge only two algorithms have been proposed for this purpose.