Paper Info
Title
Inter-Annotator Agreement on a Linguistic Ontology for Spatial Language - A Case Study for GUM-Space
Abstract
In this paper, we present a case study for measuring inter-annotator agreement on a linguistic ontology for spatial language, namely the spatial extension of the Generalized Upper Model. This linguistic ontology specifies semantic categories, and it is used in dialogue systems for natural language of space in the context of human-computer interaction and spatial assistance systems. Its core representation for spatial language distinguishes how sentences can be structured and categorized into units that contribute certain meanings to the expression. This representation is here evaluated in terms of inter-annotator agreement: four uninformed annotators were instructed by a manual how to annotate sentences with the linguistic ontology. They have been assigned to annotate 200 sentences with varying length and complexity. Their resulting agreements are calculated together with our own 'expert annotation' of the same sentences. We show that linguistic ontologies can be evaluated with respect to inter-annotator agreement, and we present encouraging results of calculating agreements for the spatial extension of the Generalized Upper Model.
Year
Venue
Keywords
2010
LREC 2010 - SEVENTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION
natural language,human computer interaction
Field
DocType
Citations 
Ontology (information science),Linguistic description,Ontology,Computer science,Object language,Natural language programming,Natural language,Natural language processing,Language identification,Universal Networking Language,Artificial intelligence,Linguistics
Conference
1
PageRank 
References 
Authors
0.45
8
1
Name
Order
Citations
PageRank
Joana Hois116811.93