Title
Typical Cases of Annotators' Disagreement in Discourse Annotations in Prague Dependency Treebank
Abstract
In this paper, we present the first results of the parallel Czech discourse annotation in the Prague Dependency Treebank 2.0. Having established an annotation scenario for capturing semantic relations crossing the sentence boundary in a discourse, and having annotated the first sections of the treebank according to these guidelines, we report now on the results of the first evaluation of these manual annotations. We give an overview of the process of the annotation itself, which we believe is to a large degree language-independent and therefore accessible to any discourse researcher. Next, we describe the inter-annotator agreement measurement, and, most importantly, we classify and analyze the most common types of annotators' disagreement and propose solutions for the next phase of the annotation. The annotation is carried out on dependency trees (on the tectogrammatical layer), this approach is quite novel and it brings us some advantages when interpreting the syntactic structure of the discourse units.
Year
Venue
Field
2010
LREC 2010 - SEVENTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION
Czech,Annotation,Computer science,Treebank,Natural language processing,Artificial intelligence,Linguistics,Sentence,Syntactic structure
DocType
Citations 
PageRank 
Conference
5
0.62
References 
Authors
3
4
Name
Order
Citations
PageRank
Sárka Zikánová1333.97
Lucie Mladová2293.41
Jirí Mírovský36516.79
Pavlína Jínová4213.56