Weighted majority vote using Shapley values in crowdsourcing - Archive ouverte HAL Access content directly
Conference Papers Year : 2024

Weighted majority vote using Shapley values in crowdsourcing

Abstract

Crowdsourcing has emerged as a pivotal paradigm for harnessing collective intelligence to solve data annotation tasks. Effective label aggregation, crucial for leveraging the diverse judgments of contributors, remains a fundamental challenge in crowdsourcing systems. This paper introduces a novel label aggregation strategy based on Shapley values, a concept originating from cooperative game theory. By integrating Shapley values as worker weights into the Weighted Majority Vote label aggregation (WMV), our proposed framework aims to address the interpretability of weights assigned to workers. This aggregation reduces the complexity of probabilistic models and the difficulty of the final interpretation of the aggregation from the workers' votes. We show improved accuracy against other WMV-based label aggregation strategies. We demonstrate the efficiency of our strategy on various real datasets to explore multiple crowdsourcing scenarios.
Fichier principal
Vignette du fichier
cap2024.pdf (1.28 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04573727 , version 1 (13-05-2024)

Identifiers

  • HAL Id : hal-04573727 , version 1

Cite

Tanguy Lefort, Benjamin Charlier, Alexis Joly, Joseph Salmon. Weighted majority vote using Shapley values in crowdsourcing. CAp 2024 - Conférence sur l'Apprentissage Automatique, Jul 2024, Lille, France. ⟨hal-04573727⟩
1 View
0 Download

Share

Gmail Mastodon Facebook X LinkedIn More