Name
Abstract | ||
|---|---|---|
In many supervised learning tasks it can be costly or infeasible to obtain objective, reliable labels. We may, however, be able to obtain a large number of subjective, possibly noisy, labels from multiple annotators. Typically, annotators have different levels of expertise (i.e., novice, expert) and there is considerable diagreement among annotators. We present a Gaussian process (GP) approach to regression with multiple labels but no absolute gold standard. The GP framework provides a principled non-parametric framework that can automatically estimate the reliability of individual annotators from data without the need of prior knowledge. Experimental results show that the proposed GP multi-annotator model outperforms models that either average the training data or weigh individually learned single-annotator models. |
Year | DOI | Venue |
|---|---|---|
2011 | 10.1007/978-3-642-21738-8_21 | ICANN (2) |
Keywords | Field | DocType |
multiple annotators,considerable diagreement,training data,multiple label,principled non-parametric framework,gaussian process,individual annotators,gp framework,proposed gp multi-annotator model,absolute gold standard,gold standard,supervised learning | Training set,Kriging,Regression,Computer science,Supervised learning,Gaussian process,Artificial intelligence,Machine learning | Conference |
Volume | ISSN | Citations |
6792 | 0302-9743 | 20 |
PageRank | References | Authors |
0.91 | 10 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Perry Groot | 1 | 175 | 17.36 |
| Adriana Birlutiu | 2 | 70 | 6.41 |
| Tom Heskes | 3 | 1519 | 198.44 |