In many machine learning applications, when we face the curse of dimensionality where the number of features is too large while the number of training examples is too small; relevant features selection is a critical need before applying any prediction models on our small data. To extract the maximum information out of the data in this setting; a possible and interesting source of useful information would be the experts of the field. The previous works introduced the benefits of querying one expert on the relevance and weights of the coefficients in sparse linear regression and how this feedback improves the prediction. But this method falls under the assumption that the voter has sufficient knowledge. Naturally, it is very hard to ensure the level of accuracy, which exposes the feedback to the risk of being biased. We define a way to calculate the accuracy of a user feedback and propose two methods to combine votes of multiple users on the relevance of features, calculate the accuracy of the final votes and study the reduction in the prediction error in a synthetic setting.