Paper Info
Title
Reply & Supply: Efficient crowdsourced exploration for growing question sets and nets.
Abstract
Crowdsourcing is now invaluable in many domains for performing data collection and analysis by distributing tasks to workers, yet the true potential of crowdsourcing lies in workers not only performing tasks or answering questions but also in using their intuition and experience to contribute new tasks or questions for subsequent crowd analysis. Algorithms to efficiently assign tasks to workers focus on fixed question sets, but exploration of a growing set of questions presents greater challenges. For example, Markov Decision Processes made significant advances to question assignment algorithms, but they do not naturally account for hidden state transitions needed to represent newly contributed questions. We consider growing question sets as growing networks of items linked by questions. If these networks grew at random, they would obey classic u0027rich-get-richeru0027 dynamics, where the number of questions associated with an item depends on how early the item entered the network. This leads to more crowd time spent answering questions related to older items and less time exploring newer items. We introduce a probability matching algorithm to curtail this bias by efficiently distributing workers between exploring new questions and addressing current questions. The method handles non-network growing question sets equally well. Experiments and simulations demonstrate that this algorithm can efficiently explore an unbounded set of questions while maintaining confidence in crowd answers.
Year
Venue
Field
2016
arXiv: Social and Information Networks
Data mining,Data collection,Computer science,Crowdsourcing,Bounded set,Intuition,Markov decision process,Artificial intelligence,Crowd analysis,Probability matching,Machine learning
DocType
Volume
Citations 
Journal
abs/1611.00954
0
PageRank 
References 
Authors
0.34
0
2
Name
Order
Citations
PageRank
Thomas C. McAndrew100.34
James P. Bagrow228126.25