Paper Info
Title
Assessment of different workflow strategies for annotating discourse relations: a case study with HDRB
Abstract
In this paper we present our experiments with different annotation workflows for annotating discourse relations in the Hindi Discourse Relation Bank(HDRB). In view of the growing interest in the development of discourse data-banks based on the PDTB framework and the complexities associated with the discourse annotation, it is important to study and analyze approaches and practices followed in the annotation process. The ultimate goal is to find an optimal balance between accurate description of discourse relations and maximal inter-rater reliability. We address the question of the choice of annotation work-flow for discourse and how it affects the consistency and hence the quality of annotation. We conduct multiple annotation experiments using different work-flow strategies, and evaluate their impact on inter-annotator agreement. Our results show that the choice of annotation work-flow has a significant effect on the annotation load and the comprehension of discourse relations for annotators, as is reflected in the inter-annotator agreement results.
Year
DOI
Venue
2013
10.1007/978-3-642-37247-6_42
CICLing
Keywords
Field
DocType
discourse relation,multiple annotation experiment,different work-flow strategy,annotation process,different workflow strategy,discourse annotation,annotating discourse relation,different annotation workflows,case study,annotation load,annotation work-flow,inter-annotator agreement
Discourse relation,Annotation,Hindi,Computer science,As is,Natural language processing,Artificial intelligence,Workflow,Comprehension
Conference
Citations 
PageRank 
References 
1
0.36
11
Authors
5
Name
Order
Citations
PageRank
Himanshu Sharma1202.58
Praveen Dakwale271.28
Dipti M. Sharma372.02
Rashmi Prasad461.15
Aravind K. Joshi53479791.99