The ever-growing and dynamic nature of user-generated data in online systems poses obvious challenges on how we learn from such data. The underlying problem is how to adapt, in real time, to multiple simultaneous changes involving individual users, user contexts and the system as a whole. Many algorithms are able to adjust their output to some of these changes in real time; however, this requires that the model has been previously trained on data with very similar phenomena. To adapt to new trends, preferences and other unpredictable phenomena, algorithms must be able to update the underlying model itself, which should preferably happen online, incrementally and in real time. This motivates the research on adaptive methods able to maintain and evolve predictive models over time. Incremental learning algorithms and data stream mining have gained maturity in recent years. However, this body of knowledge has not been applied to predictive user modeling, and although the potential to solve relevant problems is high, advances in this direction are far from trivial, calling for further research in this direction. This special issue provides contributions on the above challenges, explored and discussed in the Online Recommender Systems and User Modeling (ORSUM) workshop series that have taken place since 2018 in The Web Conference 2018 (Jorge et al. 2018) and the ACM Conference on Recommender Systems between 2019 and 2022 (Vinagre et al. 2019, 2020, 2021).