Name
Title | ||
|---|---|---|
Exploitation of an Arabic Language Resource for MT Evaluation: Using Buckwalter-based Lookup Tool to Augment CMU Alignment Algorithm |
Abstract | ||
|---|---|---|
Voss et al. (2006) analyzed newswire translations of three DARPA GALE Arabic-English MT systems at the segment level in terms of subjective judgment scores, automated metric scores, and correlations among these different score types. At this level of granularity, the correlations are weak. In this paper, we begin to reconcile the subjective and automated scores that underlie these correlations by explicitly "grounding" MT output with its Reference Translation (RT) prior to subjective or automated evaluation. The first two phases of our approach annotate {MT, RT} pairs with the same types of textual comparisons that subjects intuitively apply, while the third phase (not presented here) entails scoring the pairs: (i) automated calculation of "MT-RT hits" using CMU aligner from METEOR, (ii) an extension phase where our Buckwalter-based Lookup Tool serves to generate six other textual comparison categories on items in the MT output that the CMU aligner does not identify, and (iii) given the fully categorized RT & MT pair, a final adequacy score is assigned to the MT output, either by an automated metric based on weighted category counts and segment length, or by a trained human judge. |
Year | Venue | Keywords |
|---|---|---|
2008 | SIXTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, LREC 2008 | machine translation,arabic language |
Field | DocType | Citations |
Arabic,Computer science,Machine translation,Speech recognition,Machine translation software usability,Natural language processing,Artificial intelligence,Augment | Conference | 0 |
PageRank | References | Authors |
0.34 | 4 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Clare R. Voss | 1 | 344 | 29.51 |
| Jamal Laoudi | 2 | 29 | 4.32 |
| Jeffrey Micher | 3 | 1 | 1.40 |