Paper Info
Title
Fuzzy matching for N-gram-based MT evaluation
Abstract
N-gram-based metrics have been used widely in automatic evaluation of machine translation. However, most of them also lose merits due to the strict policy of matching of n-grams. Especially, the policy of exact matching leads to take synonyms as totally different words and thus give unreasonable estimation. This paper introduces fuzzy matching for n-grams, which refers to a semantic similarity function based on WordNet. And it is used to find a match with the highest similarity when incorporated into BLEU, the representative of n-gram-based evaluation metrics. Since WordNet can contribute more to high-order n-grams and fuzzy matching can perform well even with fewer references, experiments on MTC Part 2 (LDC2003T17) show our proposed method can greatly improve correlation between BLEU and human evaluation both at segment-level and document-level. Furthermore, BLEU incorporating fuzzy matching achieves more significant improvement at document-level evaluation.
Year
DOI
Venue
2012
10.1007/978-3-642-36337-5_5
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Keywords
Field
DocType
semantic similarity,n-gram-based metrics,human evaluation,automatic evaluation,document-level evaluation,n-gram-based mt evaluation,highest similarity,fuzzy matching,strict policy,n-gram-based evaluation metrics,exact matching,machine translation,wordnet
Semantic similarity,BLEU,Pattern recognition,Evaluation of machine translation,Computer science,Machine translation,Approximate string matching,n-gram,Artificial intelligence,WordNet
Conference
Volume
Issue
ISSN
7717 LNAI
null
16113349
Citations 
PageRank 
References 
0
0.34
11
Authors
2
Name
Order
Citations
PageRank
Liangyou Li132.74
Zhengxian Gong2698.49