Deep learning has brought significant advancements in the field of artificial intelligence, particularly in robotics, imaging, sound processing, etc. However, a common major challenge faced by all neural networks is their substantial demand for data during the learning process. The required data must be both quantitative and stationary to ensure the proper computing of standard models. Nevertheless, complying to these constraints is often impossible for many real-life applications because of dynamic environments. Indeed, modifications can occur in the distribution of the data or even in the goals to pursue within these environments. This is known as data and concept drift. Research in the field of continual learning seeks to address these challenges by implementing evolving models capable of adaptation over time. This notably involves finding a compromise on the plasticity/stability dilemma while taking into account material and computational constraints. Exploratory efforts are evident in all applications of deep learning (graphs, reinforcement learning, etc.), but to date, there is still a limited amount of work in the case of time series, specifically in the context of regression and forecasting. This paper aims to provide a first survey on this field of continuous learning applied to time series forecasting.