Paper Info
Title
Using semi-experts to derive judgments on word sense alignment: a pilot study.
Abstract
The overall goal of this project is to evaluate the performance of word sense alignment (WSA) systems, focusing on obtaining examples appropriate to language learners. Building a gold standard dataset based on human expert judgments is costly in time and labor, and thus we gauge the utility of using semi-experts in performing the annotation. In an online survey, we present a sense of a target word from one dictionary with senses from the other dictionary, asking for judgments of relatedness. We note the difficulty of agreement, yet the utility in using such results to evaluate WSA work. We find that one's treatment of related senses heavily impacts the results for WSA.
Year
Venue
Keywords
2012
LREC 2012 - EIGHTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION
word sense alignment,crowdsourcing,evaluation,WordNet,COBUILD
Field
DocType
Citations 
Annotation,Computer science,Speech recognition,Artificial intelligence,Natural language processing,Word sense
Conference
3
PageRank 
References 
Authors
0.42
11
3
Name
Order
Citations
PageRank
Soojeong Eom140.80
Markus Dickinson231.43
Graham Katz358941.68