Title
Evaluating Productivity Gains of Hybrid ASR-MT Systems for Translation Dictation
Abstract
This paper is about Translation Dictation with ASR, that is, the use of Automatic Speech Recognition (ASR) by human translators, in order to dictate translations. We are particularly interested in the productivity gains that this could provide over conventional keyboard input, and ways in which such gains might be increased through a combination of ASR and Statistical Machine Translation (SMT). In this hybrid technology, the source language text is presented to both the human translator and a SMT system. The latter produces N- best translations hypotheses, which are then used to fine tune the ASR language model and vocabulary towards utterances which are probable translations of source text sentences. We conducted an ergonomic experiment with eight professional translators dictating into French, using a top of the line off- the-shelf ASR system (Dragon NatuallySpeaking 8). We found that the ASR system had an average Word Error Rate (WER) of 11.7%, and that translation using this system did not provide statistically significant productivity increases over keyboard input, when following the manufacturer recommended procedure for error correction. However, we found indications that, even in its current imperfect state, French ASR might be beneficial to translators who are already used to dictation (either with ASR or a dictaphone), but more focused experiments are needed to confirm this. We also found that dictation using an ASR with WER of 4% or less would have resulted in statistically significant (p < 0.6) productivity gains in the order of 25.1% to 44.9% Translated Words Per Minute. We also evaluated the extent to which the limited manufacturer provided Domain Adaptation features could be used to positively bias the ASR using SMT hypotheses. We found that the relative gains in WER were much lower than has been reported in the literature for tighter integration of SMT with ASR, pointing the advantages of tight integration approaches and the need for more research in that area.
Year
Venue
Keywords
2008
IWSLT
error correction,language model,word error rate,statistical significance,automatic speech recognition
Field
DocType
Citations 
Words per minute,Computer science,Machine translation,Word error rate,Error detection and correction,Speech recognition,Dictation,Artificial intelligence,Natural language processing,Source text,Vocabulary,Language model
Conference
1
PageRank 
References 
Authors
0.38
9
5
Name
Order
Citations
PageRank
Alain Désilets113615.99
Marta Stojanovic2161.54
Jean-François Lapointe3306.89
Rick Rose410.38
Aarthi Reddy5232.27