Paper Info
Title
Collaborative Development and Evaluation of Text-processing Workflows in a UIMA-supported Web-based Workbench.
Abstract
Challenges in creating comprehensive text-processing worklows include a lack of the interoperability of individual components coming from different providers and/or a requirement imposed on the end users to know programming techniques to compose such workflows. In this paper we demonstrate Argo, a web-based system that addresses these issues in several ways. It supports the widely adopted Unstructured Information Management Architecture (UIMA), which handles the problem of interoperability; it provides a web browser-based interface for developing workflows by drawing diagrams composed of a selection of available processing components; and it provides novel user-interactive analytics such as the annotation editor which constitutes a bridge between automatic processing and manual correction. These features extend the target audience of Argo to users with a limited or no technical background. Here, we focus specifically on the construction of advanced workflows, involving multiple branching and merging points, to facilitate various comparative evalutions. Together with the use of user-collaboration capabilities supported in Argo, we demonstrate several use cases including visual inspections, comparisions of multiple processing segments or complete solutions against a reference standard, inter-annotator agreement, and shared task mass evaluations. Ultimetely, Argo emerges as a one-stop workbench for defining, processing, editing and evaluating text processing tasks.
Year
Venue
Keywords
2012
LREC 2012 - EIGHTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION
Text mining,Text-processing workflows,Evaluation methods
Field
DocType
Citations 
Information management,Use case,Software engineering,End user,Interoperability,Computer science,Artificial intelligence,Natural language processing,Web application,Analytics,Workflow,Text processing
Conference
3
PageRank 
References 
Authors
0.44
4
3
Name
Order
Citations
PageRank
Rafal Rak138218.30
Andrew Rowley21058.87
Sophia Ananiadou32658183.08