Paper Info
Title
Crowd Guilds: Worker-led Reputation and Feedback on Crowdsourcing Platforms.
Abstract
Crowd workers are distributed and decentralized. While decentralization is designed to utilize independent judgment to promote high-quality results, it paradoxically undercuts behaviors and institutions that are critical to high-quality work. Reputation is one central example: crowdsourcing systems depend on reputation scores from decentralized workers and requesters, but these scores are notoriously inflated and uninformative. In this paper, we draw inspiration from historical worker guilds (e.g., in the silk trade) to design and implement crowd guilds: centralized groups of crowd workers who collectively certify each other's quality through double-blind peer assessment. A two-week field experiment compared crowd guilds to a traditional decentralized crowd work model. Crowd guilds produced reputation signals more strongly correlated with ground-truth worker quality than signals available on current crowd working platforms, and more accurate than in the traditional model.
Year
DOI
Venue
2017
10.1145/2998181.2998234
CSCW
Keywords
DocType
Volume
crowdsourcing platforms, human computation
Conference
abs/1611.01572
Citations 
PageRank 
References 
2
0.36
29
Authors
28
Name
Order
Citations
PageRank
Mark A. Whiting114941.20
Dilrukshi Gamage2326.53
Snehalkumar (Neil) S. Gaikwad3313.04
Aaron Gilbee4181.68
Shirish Goyal5182.35
Alipta Ballav6181.68
Dinesh Majeti7181.68
Nalin Chhibber8182.69
Angela Richmond-Fuller9182.02
Freddie Vargus1020.36
Tejas Seshadri Sarma1120.36
Varshine Chandrakanthan1220.36
Teogenes Moura1330.77
Mohamed Hashim Salih1420.36
Gabriel Bayomi Tinoco Kalejaiye1520.36
Adam Ginzberg16302.70
Catherine A. Mullings1720.36
Yoni Dayan1820.36
Kristy Milland19764.57
Henrique Orefice2020.36
Jeff Regino21181.68
Sayna Parsi2220.36
Kunz Mainali2320.36
Vibhor Sehgal24181.68
Sekandar Matin25181.68
Akshansh Sinha2641.05
Rajan Vaish2741.05
Michael S. Bernstein2820.36