Paper Info
Title
Large Scale Multilingual Broadcast Data Collection to Support Machine Translation and Distillation Technology Development
Abstract
The development of technologies to address machine translation and distillation of multilingual broadcast data depends heavily on the collection of large volumes of material from modern data providers. To address the needs of GALE researchers, the Linguistic Data Consortium (LDC) developed a system for collecting broadcast news and conversation from a variety of Arabic, Chinese and English broadcasters. The system is highly automated, easily extensible and robust and is capable of collecting, processing and evaluating hundreds of hours of content from several dozen sources per day. In addition to this extensive system, LDC manages three remote collection sites to maximize the variety of available broadcast data and has designed a portable broadcast collection platform to facilitate remote collection. This paper will present a detailed a description of the design and implementation of LDC's collection system, the technical challenges and solutions to large scale broadcast data collection efforts and an overview of the system's operation. This paper will also discuss the challenges of managing remote collections, in particular, the strategies used to normalize data formats, naming conventions and delivery methods to achieve optimal integration of remotely-collected data into LDC's collection database and downstream tasking workflow.
Year
Venue
Keywords
2010
LREC 2010 - SEVENTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION
machine translation,data collection
Field
DocType
Citations 
Broadcasting,Linguistic Data Consortium,Satellite dish,TRECVID,Computer science,Machine translation,NIST,Natural language processing,Artificial intelligence,Text Retrieval Conference,Workflow
Conference
1
PageRank 
References 
Authors
0.38
1
3
Name
Order
Citations
PageRank
Kevin Walker16521.51
Christopher Caruso2232.12
Denise DiPersio374.96