This paper introduces an adaptative segmentation system that was designed for color document image analysis. The method is based on the serialization of a k-means algorithm that is applied sequentially by using a sliding window over the image. During the window’s displacement, the algorithm reuses information from the clusters computed in the previous window and automatically adjusts them in order to adapt the classifier to any new local variation of the colors. To improve the results, we propose to define several different clusters in the color feature space for each logical class. We also reintroduce the user into the initialization step to define the number of classes and the different samples for each class. This method has been tested successfully on ancient color manuscripts, video images and multiple natural and non-natural images having heavy defects and showing illumination variation and transparency. The proposed algorithm is generic enough to be applied on a large variety of images for different purposes such as color image segmentation as well as binarization.