Name
Abstract | ||
|---|---|---|
Amazon's Mechanical Turk service has been successfully applied to many natural language processing tasks. However, the task of named entity recognition presents unique challenges. In a large annotation task involving over 20,000 emails, we demonstrate that a competitive bonus system and inter-annotator agreement can be used to improve the quality of named entity annotations from Mechanical Turk. We also build several statistical named entity recognition models trained with these annotations, which compare favorably to similar models trained on expert annotations. |
Year | Venue | Keywords |
|---|---|---|
2010 | Mturk@HLT-NAACL | large email datasets,mechanical turk,expert annotation,entity recognition model,large annotation task,entity recognition,competitive bonus system,natural language processing task,mechanical turk service,inter-annotator agreement,entity annotation |
Field | DocType | Citations |
Annotation,Computer science,Named entity,Natural language processing,Artificial intelligence,Named-entity recognition | Conference | 18 |
PageRank | References | Authors |
0.86 | 11 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Nolan Lawson | 1 | 18 | 1.20 |
| Kevin Eustice | 2 | 58 | 6.51 |
| Mike Perkowitz | 3 | 1350 | 189.27 |
| Meliha Yetisgen-Yildiz | 4 | 328 | 34.25 |