Nowadays, the biomedical field takes advantage of computer science and biotechnologies development to look for potential new strategies in the fight against complex diseases like cancer. In cancer stage prediction and prognosis, researchers can look for biomarkers using machine learning (ML) approaches able to perform pattern recognition. Unfortunately, even with an expertise in ML field, it is difficult to know which algorithm will perform best on a specific type of data. ML-based strategies have multiple steps that can be difficult to set up as it implies the tuning of numerous parameters at every step. Thus, ML based studies usually focus on a unique model that is expected to fit their specific research question. However, based on such strategy, variations in the methods may lead to completely different results and model generalization is limited. In this study, we make a first contribution towards a large scale analysis meant to understand the behavior pattern of ML methods in the context of biomarker signature identification from omics cancer data. We present preliminary results of colorectal cancer stage prediction based on the intra-and inter-group comparisons of two types of ML methods: Bayes-based and Trees. We first estimate the robustness of various Bayes and Tree based models with respect to the tuning of their parameters. We then analyze the composition of the produced signatures in order to assess their level of confidence, by looking for consistent features between models. Preliminary results suggest that Bayes-based algorithms are promising as their performances and signatures seem to be consistent across various configurations.