Name
Abstract | ||
|---|---|---|
This study addresses problems of reliability in the creation of tagged corpora by self-selected semi-anonymous raters. In order to account for both strong and weak raters, this paper contributes a recursive technique for scoring rater reliability. By assigning raters trust scores in the proposed method, candidate labels can be weighted by a confidence score and low-confidence ratings can be routed to an expert rater or additional amateur raters for further action. |
Year | DOI | Venue |
|---|---|---|
2012 | 10.1145/2232817.2232885 | JCDL |
Keywords | Field | DocType |
low-confidence rating,candidate label,rater reliability,additional amateur raters,iterative reliability measure,confidence score,weak raters,self-selected semi-anonymous raters,semi-anonymous annotators,raters trust score,expert rater,crowdsourcing,reliability,annotation | Confidence score,Annotation,Information retrieval,Computer science,Crowdsourcing,Amateur,Recursion | Conference |
Citations | PageRank | References |
1 | 0.36 | 3 |
Authors | ||
1 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Peter Organisciak | 1 | 67 | 10.31 |