The present research aims at improving the accuracy of labels on Petri dish images containing Colony Forming Units using Artificial Intelligence algorithms. Indeed, the labeling methods proposed by classical computer vision software such as ScanStation for example, are prone to errors and the manual correction of these errors is a difficult task. We propose a methodology based on AI models. At first, a YOLO model is trained on the existing labels given by ScanStation. The bounding boxes provided by ScanStation and YOLO are then binarized using the OTSU algorithm to generate semantic labels that are used to train a U-Net. Then, a Xception model is trained to classify all the segments generated by the U-Net as either outlier or colony. For new data, the trained U-Net and Xception models are used to improve the labeling. The results indicate that the proposed approach improves the accuracy of the labeling process without human correction.