Paper Info
Title
A Methodology for a Semi-Automatic Evaluation of the Lexicons of Machine Translation Systems
Abstract
The lexicon is a major part of any Machine Translation (MT) system. If the lexicon of an MT system is not adequate, this will affect the quality of the whole system. Building a comprehensive lexicon, i.e., one with a high lexical coverage, is a major activity in the process of developing a good MT system. As such, the evaluation of the lexicon of an MT system is clearly a pivotal issue for the process of evaluating MT systems. In this paper, we introduce a new methodology that was devised to enable developers and users of MT Systems to evaluate their lexicons semi-automatically. This new methodology is based on the idea of the importance of a specific word or, more precisely, word sense, to a given application domain. This importance, or weight, determines how the presence of such a word in, or its absence from, the lexicon affects the MT system's lexical quality, which in turn will naturally affect the overall output quality. The method, which adopts a black-box approach to evaluation, was implemented and applied to evaluating the lexicons of three commercialEnglish–Arabic MT systems. A specific domain was chosen in which the various word-sense weights were determined by feeding sample texts from the domain into a system developed specifically for that purpose. Once this database of word senses and weights was built, test suites were presented to each of the MT systems under evaluation and their output rated by a human operator as either correct or incorrect. Based on this rating, an overall automated evaluation of the lexicons of the systems was deduced.
Year
DOI
Venue
2001
10.1023/A:1014504808954
Machine Translation
Keywords
Field
DocType
arabic,translation quality.,word sense,machine translation systems,overall automated evaluation,arabic mt system,evaluation,mt systems,good mt system,specific word,new methodology,coverage,mt system,lexicons,whole system,comprehensive lexicon,semi-automatic evaluation,machine translation
Human operator,Computer science,Computational linguistics,Machine translation,Lexicon,Natural language processing,Transfer-based machine translation,Artificial intelligence,Application domain,Electronic dictionary,Linguistics,Applied linguistics
Journal
Volume
Issue
ISSN
16
2
1573-0573
Citations 
PageRank 
References 
2
0.47
7
Authors
2
Name
Order
Citations
PageRank
Ahmed Guessoum1114.97
Rached Zantout2187.67