Kernel discriminant analysis (KDA), the nonlinear extension of linear Discriminant Analysis (LDA), is a popular tool for learning one or multiple categories in nonlinear data sets. However, in most modern pattern recognition applications such as video surveillance, data are collected in flow and require sequential processing. In this context, KDA is faced two critical issues: an original formulation unsuited to the dynamic nature of the data and an increasing memory requirement for the kernel matrix storage. Motivated by the state-of-the-art performance reported by the null KDA, we propose in this paper a new solution to solve the null KDA (NKDA) in the context of data streams. Compared to previous works, our contribution is based on three points: first, we develop an exact incremental scheme which guarantees accurate solutions. Secondly, we develop a compression mechanism based on the following observation: rger the size of the training data set more the distances in the null space contract This property of the null space leads to formulate an indicator of redundancy in the training data set. This criterion is the cornerstone of our incremental KNDA because it authorizes incremental learning on large-scale data sets. Third, the problem of novelty detection in multi-class and one-class scenarios is addressed. More precisely, the fact that distances in the null space change over the training period leads us to define adjustable novelty thresholds. Lastly, numerous experiments based on various publicly available data sets and state-of-the-art classifiers show that the proposed method is effective both for multi-class and one-class real applications.