Name
Abstract | ||
|---|---|---|
With the increasing popularity of online crowdsourcing platforms such as Amazon Mechanical Turk (AMT), building supervised learning models for datasets with multiple annotators is receiving an increasing attention from researchers. These platforms provide an inexpensive and accessible resource that can be used to obtain labeled data, and in many situations the quality of the labels competes directly with those of experts. For such reasons, much attention has recently been given to annotator-aware models. In this paper, we propose a new probabilistic model for supervised learning with multiple annotators where the reliability of the different annotators is treated as a latent variable. We empirically show that this model is able to achieve state of the art performance, while reducing the number of model parameters, thus avoiding a potential overfitting. Furthermore, the proposed model is easier to implement and extend to other classes of learning problems such as sequence labeling tasks. |
Year | DOI | Venue |
|---|---|---|
2013 | 10.1016/j.patrec.2013.05.012 | Pattern Recognition Letters |
Keywords | Field | DocType |
random labelers,accessible resource,multiple annotators,new probabilistic model,increasing attention,increasing popularity,model parameter,amazon mechanical turk,supervised learning,different annotators,expectation maximization,logistic regression,crowdsourcing | Sequence labeling,Pattern recognition,Expectation–maximization algorithm,Crowdsourcing,Computer science,Popularity,Latent variable,Supervised learning,Artificial intelligence,Statistical model,Overfitting,Machine learning | Journal |
Volume | Issue | ISSN |
34 | 12 | 0167-8655 |
Citations | PageRank | References |
28 | 1.34 | 15 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Filipe Rodrigues | 1 | 97 | 8.80 |
| Francisco C. Pereira | 2 | 331 | 33.07 |
| Bernardete Ribeiro | 3 | 758 | 82.07 |