Name
Abstract | ||
|---|---|---|
Predictive models play a key role for inference and decision making in crowdsourcing. We present methods that can be used to guide the collection of data for enhancing the competency of such predictive models while using the models to provide a base crowdsourcing service. We focus on the challenge of ideally balancing the goals of collecting data over time for learning and for improving task performance with the cost of workers' contributions over the lifetime of the operation of a system. We introduce the use of distributions over a set of predictive models to represent uncertainty about the dynamics of the world. We employ a novel Monte Carlo algorithm to reason simultaneously about uncertainty about the world dynamics and the progression of task solution as workers are hired over time to optimize hiring decisions. We evaluate the methodology with experiments on a challenging citizen-science problem, demonstrating how it balances exploration and exploitation over the lifetime of a crowdsourcing system. |
Year | Venue | Keywords |
|---|---|---|
2013 | IJCAI | challenging citizen-science problem,task solution,present method,task performance,crowdsourcing system,novel monte carlo algorithm,world dynamic,key role,predictive model,pomdp,adaptive control,crowdsourcing,bayesian inference |
Field | DocType | Citations |
Data science,Competence (human resources),Bayesian inference,Crowdsourcing,Partially observable Markov decision process,Computer science,Inference,Wisdom of the crowd,Artificial intelligence,Adaptive control,Lifelong learning,Machine learning | Conference | 11 |
PageRank | References | Authors |
0.57 | 18 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Ece Kamar | 1 | 577 | 48.11 |
| Ashish Kapoor | 2 | 1833 | 119.72 |
| Eric Horvitz | 3 | 9402 | 1058.25 |