Name
Title | ||
|---|---|---|
Improving Statistical Machine Translation Using Bayesian Word Alignment and Gibbs Sampling |
Abstract | ||
|---|---|---|
We present a Bayesian approach to word alignment inference in IBM Models 1 and 2. In the original approach, word translation probabilities (i.e., model parameters) are estimated using the expectation-maximization (EM) algorithm. In the proposed approach, they are random variables with a prior and are integrated out during inference. We use Gibbs sampling to infer the word alignment posteriors. The inferred word alignments are compared against EM and variational Bayes (VB) inference in terms of their end-to-end translation performance on several language pairs and types of corpora up to 15 million sentence pairs. We show that Bayesian inference outperforms both EM and VB in the majority of test cases. Further analysis reveals that the proposed method effectively addresses the high-fertility rare word problem in EM and unaligned rare word problem in VB, achieves higher agreement and vocabulary coverage rates than both, and leads to smaller phrase tables. |
Year | DOI | Venue |
|---|---|---|
2013 | 10.1109/TASL.2013.2244087 | Audio, Speech, and Language Processing, IEEE Transactions |
Keywords | Field | DocType |
belief networks,expectation-maximisation algorithm,inference mechanisms,language translation,natural language processing,probability,random processes,sampling methods,word processing,Bayesian word alignment posterior inference,EM,Gibbs sampling,IBM models,VB,corpora types,expectation-maximization algorithm,high-fertility rare word problem,language pairs,phrase tables,random variables,sentence pairs,statistical machine translation performance improvement,unaligned rare word problem,variational Bayes inference,vocabulary coverage rates,Bayesian methods,Gibbs sampling,statistical machine translation (SMT),word alignment | Frequentist inference,Bayesian inference,Language translation,Computer science,Natural language processing,Artificial intelligence,Bayesian statistics,Word processing,Gibbs sampling,Pattern recognition,Inference,Word error rate,Speech recognition | Journal |
Volume | Issue | ISSN |
21 | 5 | 1558-7916 |
Citations | PageRank | References |
5 | 0.58 | 38 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Coskun Mermer | 1 | 5 | 0.58 |
| Murat Saraclar | 2 | 669 | 62.91 |
| Ruhi Sarikaya | 3 | 698 | 64.49 |