Filtering with the Crowd: CrowdScreen revisited (technical report) - Archive ouverte HAL Access content directly
Conference Papers Year : 2016

Filtering with the Crowd: CrowdScreen revisited (technical report)

Abstract

Filtering a set of items, based on a set of properties that can be verified by humans, is a common application of CrowdSourcing. When the workers are error-prone, each item is presented to multiple users, to limit the probability of misclassification. Since the Crowd is a relatively expensive resource, minimizing the number of questions per item may naturally result in big savings. Several algorithms to address this minimization problem have been presented in the CrowdScreen framework by Parameswaran et al. However, those algorithms do not scale well and therefore cannot be used in scenarios where high accuracy is required in spite of high user error rates. The goal of this paper is thus to devise algorithms that can cope with such situations. To achieve this, we provide new theoretical insights to the problem, then use them to develop a new efficient algorithm. We also propose novel optimizations for the algorithms of CrowdScreen that improve their scalability. We complement our theoretical study by an experimental evaluation of the algorithms on a large set of synthetic parameters as well as real-life crowdsourcing scenarios, demonstrating the advantages of our solution.
Fichier principal
Vignette du fichier
glmm-icdt16-long.pdf (992.73 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01239458 , version 1 (07-12-2015)

Licence

Attribution - CC BY 4.0

Identifiers

Cite

Benoît Groz, Ezra Levin, Isaac Meilijson, Tova Milo. Filtering with the Crowd: CrowdScreen revisited (technical report). 19th International Conference on Database Theory (ICDT 2016), Mar 2016, Bordeaux, France. pp.12:1--12:18, ⟨10.4230/LIPIcs.ICDT.2016.12⟩. ⟨hal-01239458⟩
494 View
164 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More