Several image test suites are available in the literature to evaluate the performance of classification schemes. In the framework of colour texture classification, OuTex-TC-00013 (OuTex) and Contrib-TC-00006 (VisTex) are often used. These colour texture image sets have allowed the accuracies reached by many classification schemes to be compared. However, by analysing the classification results obtained with these two sets of colour texture images, we have noticed that the use of colour histogram yields a higher rate of well-classified images compared to colour texture features. It does not take into account any texture information in the image, this incoherence leads us to question the relevance of these two benchmark colour texture sets for measuring the performances of colour texture classification algorithms. Indeed, the partitioning used to build these two sets consists of extracting training and validating sub-images of an original image. We show that such partitioning leads to biased classification results when it is combined with a classifier such as the nearest neighbour. In this paper a new relevant image test suite is proposed for evaluating colour texture classification schemes. The training and the validating sub-images come from different original images in order to ensure that the correlation of the colour texture images is minimized.