Paper Info
Title
The CALBC Silver Standard Corpus for Biomedical Named Entities - A Study in Harmonizing the Contributions from Four Independent Named Entity Taggers
Abstract
The production of gold standard corpora is time-consuming and costly. We propose an alternative: the, silver standard corpus' (SSC), a corpus that has been generated by the harmonisation of the annotations that have been delivered from a selection of annotation systems. The systems have to share the type system for the annotations and the harmonisation solution has use a suitable similarity measure for the pair-wise comparison of the annotations. The annotation systems have been evaluated against the harmonised set (630.324 sentences, 15,956,841 tokens). We can demonstrate that the annotation of proteins and genes shows higher diversity across all used annotation solutions leading to a lower agreement against the harmonised set in comparison to the annotations of diseases and species. An analysis of the most frequent annotations from all systems shows that a high agreement amongst systems leads to the selection of terms that are suitable to be kept in the harmonised set. This is the first large-scale approach to generate an annotated corpus from automated annotation systems. Further research is required to understand, how the annotations from different systems have to be combined to produce the best annotation result for a harmonised corpus.
Year
Venue
Field
2010
LREC 2010 - SEVENTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION
Annotation,Similarity measure,Computer science,Named entity,Natural language processing,Artificial intelligence
DocType
Citations 
PageRank 
Conference
8
0.88
References 
Authors
8
11
Name
Order
Citations
PageRank
dietrich rebholzschuhmann1102375.06
Antonio José Jimeno-Yepes2452.16
Erik M. Van Mulligen363344.63
Ning Kang4704.36
Jan Kors5815.64
David Milward619627.51
Peter Corbett717613.18
Ekaterina Buyko816511.45
Katrin Tomanek926015.77
Elena Beisswanger1022316.00
Udo Hahn1193788.14