In this paper, we derive an adaptive one-class classification algorithm. We propose a least-squares formulation of the problem, where the model complexity is controlled by a parsimony criterion. We consider the linear approximation criterion, and we couple it with a simple adaptive updating algorithm for online learning. We conduct experiments on synthetic datasets and real time series, and illustrate the relevance of the proposed method over existing methods, and show its low computational cost.