Digital ecosystems rely on reputation systems in order to build trust and to foster collaborations among users. Reputation systems are commonplace in the C2C and B2C contexts, however, they have not yet found mainstream acceptance in B2B environments. Our first contribution in this paper is to identify the particularities of feedback collection in B2B reputation systems. An issue that we identify is that the reputation target in the B2B context is a business, which requires evaluation on a large number of criteria. We observe that due to the wide variation in user expertise, feedback forms that require users to evaluate all criteria have significant negative consequences for rating accuracy. Our second contribution is to propose an expertise prediction algorithm for B2B reputation systems, which filters the criteria describing the target business such that each user rates only on those criteria that he has expertise in. Experiments based on our real dataset show that the algorithm accurately predicts the expertise of users in given criteria. The algorithm may also increase the motivation of users to submit feedback as well as the confidence of users in B2B reputation systems.