Name
Abstract | ||
|---|---|---|
We explore a new way to collect human annotated relations in text using Amazon Mechanical Turk. Given a knowledge base of relations and a corpus, we identify sentences which mention both an entity and an attribute that have some relation in the knowledge base. Each noisy sentence/relation pair is presented to multiple turkers, who are asked whether the sentence expresses the relation. We describe a design which encourages user efficiency and aids discovery of cheating. We also present results on inter-annotator agreement. |
Year | Venue | Keywords |
|---|---|---|
2010 | Mturk@HLT-NAACL | non-expert correction,noisy sentence,aids discovery,relation pair,present result,knowledge base,user efficiency,relation annotation,human annotated relation,amazon mechanical turk,multiple turkers,inter-annotator agreement |
Field | DocType | Citations |
Information retrieval,Computer science,Artificial intelligence,Natural language processing,Knowledge base,Cheating,Sentence,Machine learning | Conference | 7 |
PageRank | References | Authors |
0.68 | 5 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Matthew Gormley | 1 | 84 | 10.25 |
| Adam Gerber | 2 | 74 | 3.95 |
| Mary Harper | 3 | 258 | 20.54 |
| Mark Dredze | 4 | 3092 | 176.22 |