---
title: "Phrasetable Smoothing for Statistical Machine Translation (2006)"
description: "The authors present what they describe as the first systematic comparison of phrasetable… 0.45 BLEU. 22 references, cited by 63 papers here."
authors: ["George Foster", "Roland Kuhn", "Howard Johnson"]
year: 2006
venue: "Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/W06-1607/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/foster-2006-phrasetable-smoothing-statistical-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/foster-2006-phrasetable-smoothing-statistical-machine-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2006](https://md.research.elamur.ai/free/en/papers/2006/) / Phrasetable Smoothing for Statistical Machine Translation

# Phrasetable Smoothing for Statistical Machine Translation

**Authors:** [George Foster](https://md.research.elamur.ai/free/en/authors/george-foster/), [Roland Kuhn](https://md.research.elamur.ai/free/en/authors/roland-kuhn/) and [Howard Johnson](https://md.research.elamur.ai/free/en/authors/howard-johnson/) · **Year:** 2006 · **Venue:** [Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2006](https://md.research.elamur.ai/free/en/venues/emnlp/2006/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W06-1607/) · [PDF](https://aclanthology.org/W06-1607.pdf) · [OpenAlex](https://openalex.org/W2117827367)

**Citations:** 111 citations (OpenAlex, 2026-10-14) · [63](https://md.research.elamur.ai/free/en/papers/foster-2006-phrasetable-smoothing-statistical-machine-translation/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Phrase-based SMT systems usually estimate phrase translation probabilities with plain relative frequencies (RF). This overfits: a pair whose phrases each appear only once gets probability 1, which beats better-supported pairs during decoding. The authors present what they describe as the first systematic comparison of phrasetable smoothing methods. They split these into two families. "Black-box" methods treat phrases as atomic units and adapt LM-style techniques such as Good-Turing, fixed-discount and Kneser-Ney. "Glass-box" methods decompose phrases into words, as in Zens-Ney noisy-or and Koehn-Och-Marcu IBM1 smoothing. They also test combinations of the two families.

They evaluated on six WMT06 Europarl directions with small corpora and on large-scale NIST MT05 Chinese-English. Every smoothing method beat RF. In the European setting, Kneser-Ney combined with Zens-Ney gave gains of roughly 1–2 BLEU. For Chinese-English, Good-Turing and glass-box combinations did best. The study shows that a cheap, often ignored step yields consistent translation gains.

## Key points
- **Data:**
  - WMT06 Europarl: about 700,000 sentence pairs each for French, Spanish and German with English, plus a 2000-sentence dev set.
  - NIST MT05 Chinese-English: 4.98M UN sentences and 3.16M non-UN sentences, each used for a separate phrasetable. A Gigaword subset of 11.7M sentences was added for the LM.
- **Significance testing:** 1000-fold pairwise bootstrap resampling at 95% confidence.
- **European results:**
  - Any smoothing improved over RF by at least 0.45 BLEU.
  - Kneser-Ney beat Good-Turing by at least 0.30 BLEU.
  - KN3+ZN-IBM1 scored best, for example fr→en 25.35 → 27.25 and en→fr 27.20 → 29.00.
- **Chinese-English results:**
  - RF scored 29.85 and GT scored 30.66, statistically tied with KN3 at 30.54.
  - The best scores came from RF+ZN-IBM1 (30.95) and GT+KOM-IBM1 (30.81).
  - Glass-box methods used alone did poorly: ZN-IBM1 scored 29.55 and KOM-IBM1 scored 28.09.
- **Caveat:** glass-box weights were tuned for BLEU with Och's algorithm, while black-box linear weights were set heuristically. This gave glass-box methods an advantage in the comparison.

## Interesting ideas
- **Treat a phrase pair like a bigram.** Doing so lets well-understood LM smoothing (GT, KN) be reused directly on a new kind of count table. This is handy whenever you have sparse co-occurrence counts but no natural lower-order model.
- **Prefer interpolation over backoff when unseen events never matter.** The decoder only queries pairs that were seen, so the main benefit of smoothing is reshaping estimates for observed items. Interpolation does this; backoff does not. The same logic applies to any lookup-only probability table.
- **Loglinear mixing with a uniform distribution is not smoothing.** A simple Bernoulli example shows it keeps 0 and 1 intact and pushes middle values toward the extremes. This is a useful sanity check before calling a loglinear combination "smoothing."
- **Combine both families.** Black-box and glass-box smoothing appear to be complementary, so stacking a count-based discount with a word-level lexical model can beat either one alone.
- **Use learning curves for size-dependent effects.** When a method helps in one data regime but not another, training on corpus subsets can reveal whether size is the cause. Here the curves also exposed high sensitivity to which corpus sample was used.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National Research Council Canada](https://md.research.elamur.ai/free/en/government/national-research-council-canada/) — Government body, Ottawa, Canada · 40 works · 20,943 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: George Foster, Roland Kuhn and Howard Johnson. 2006. Phrasetable Smoothing for Statistical Machine Translation. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{foster2006phrasetable,
  title = {{Phrasetable Smoothing for Statistical Machine Translation}},
  author = {George Foster, Roland Kuhn and Howard Johnson},
  booktitle = {Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing},
  year = {2006},
  doi = {10.3115/1610075.1610084},
  url = {https://doi.org/10.3115/1610075.1610084}
}
```

- [← previous Discriminant-Based MRS Banking](https://md.research.elamur.ai/free/en/papers/oepen-2006-discriminant-based-mrs-banking/)
- [next → Automatic Identification of Non-Compositional Multi-Word Expressions using Latent Semantic Analysis](https://md.research.elamur.ai/free/en/papers/katz-2006-automatic-identification-non-compositional-multi/)
- [↑ back to papers of 2006](https://md.research.elamur.ai/free/en/papers/2006/)

## References

22 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Peter F. Brown, Stephen A. Della Pietra, Vincent Della J. Pietra, and Robert L. Mercer. 1993. The mathematics of Machine Translation: Parameter estimation. Computational Linguistics, 19(2):263– 312, June. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Stephen+A%2E+Della+Pietra%2C+Vincent+Della+J%2E+Pietra%2C+and+Robert+L%2E+Mercer%2E+1993%2E+The+mathematics+of+Machine+Translation%3A+Parameter+estimation%2E+Computational+Linguistics%2C+19%282%29%3A263%E2%80%93+312%2C+J)
2. M. Cettollo, M. Federico, N. Bertoldi, R. Cattoni, and B. Chen. 2005. A look inside the ITC-irst SMT system. In Proceedings of MT Summit X, Phuket, Thailand, September. International Association for Machine Translation. [find](https://scholar.google.com/scholar?q=M%2E+Cettollo%2C+M%2E+Federico%2C+N%2E+Bertoldi%2C+R%2E+Cattoni%2C+and+B%2E+Chen%2E+2005%2E+A+look+inside+the+ITC%2Dirst+SMT+system%2E+In+Proceedings+of+MT+Summit+X%2C+Phuket%2C+Thailand%2C+September%2E+International+Association+for+M)
3. Stanley F. Chen and Joshua T. Goodman. 1998. [An empirical study of smoothing techniques for language modeling](https://md.research.elamur.ai/free/en/papers/chen-1996-empirical-study-smoothing-techniques-language/). Technical Report TR-10-98, Computer Science Group, Harvard University.
4. K. Church and W. Gale. 1991. A comparison of the enhanced Good-Turing and deleted estimation methods for estimating probabilities of English bigrams. Computer speech and language, 5(1):19–54. [find](https://scholar.google.com/scholar?q=K%2E+Church+and+W%2E+Gale%2E+1991%2E+A+comparison+of+the+enhanced+Good%2DTuring+and+deleted+estimation+methods+for+estimating+probabilities+of+English+bigrams%2E+Computer+speech+and+language%2C+5%281%29%3A19%E2%80%9354%2E)
5. M. Collins and J. Brooks. 1995. [Prepositional phrase attachment through a backed-off model](https://md.research.elamur.ai/free/en/papers/collins-1995-prepositional-phrase-attachment-through-backed/). In Proceedings of the 3rd ACL Workshop on Very Large Corpora (WVLC), Cambridge, Massachusetts.
6. Ismael Garcı́a-Varea, Francisco Casacuberta, and Hermann Ney. 1998. An iterative, DP-based search algorithm for statistical machine translation. In Proceedings of the 5th International Conference on Spoken Language Processing (ICSLP) 1998, volume 4, pages 1135–1138, Sydney, Australia, December. [find](https://scholar.google.com/scholar?q=Ismael+Garc%C4%B1%CC%81a%2DVarea%2C+Francisco+Casacuberta%2C+and+Hermann+Ney%2E+1998%2E+An+iterative%2C+DP%2Dbased+search+algorithm+for+statistical+machine+translation%2E+In+Proceedings+of+the+5th+International+Conference+on+S)
7. Joshua Goodman. 2001. A bit of progress in language modeling. Computer Speech and Language. [find](https://scholar.google.com/scholar?q=Joshua+Goodman%2E+2001%2E+A+bit+of+progress+in+language+modeling%2E+Computer+Speech+and+Language%2E)
8. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2001. The Elements of Statistical Learning. Springer. [find](https://scholar.google.com/scholar?q=Trevor+Hastie%2C+Robert+Tibshirani%2C+and+Jerome+Friedman%2E+2001%2E+The+Elements+of+Statistical+Learning%2E+Springer%2E)
9. Reinhard Kneser and Hermann Ney. 1995. Improved backing-off for m-gram language modeling. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 1995, pages 181–184, Detroit, Michigan. IEEE. [find](https://scholar.google.com/scholar?q=Reinhard+Kneser+and+Hermann+Ney%2E+1995%2E+Improved+backing%2Doff+for+m%2Dgram+language+modeling%2E+In+Proceedings+of+the+International+Conference+on+Acoustics%2C+Speech%2C+and+Signal+Processing+%28ICASSP%29+1995%2C+page)
10. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Eduard Hovy, editor, Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 127–133, Edmonton, Alberta, Canada, May.… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Franz+Josef+Och%2C+and+Daniel+Marcu%2E+2003%2E+Statistical+phrase%2Dbased+translation%2E+In+Eduard+Hovy%2C+editor%2C+Proceedings+of+the+Human+Language+Technology+Conference+of+the+North+American+Chap)
11. P. Koehn, A. Axelrod, A. B. Mayne, C. Callison-Burch, M. Osborne, D. Talbot, and M. White. 2005. Edinburgh system description for the 2005 NIST MT evaluation. In Proceedings of Machine Translation Evaluation Workshop. [find](https://scholar.google.com/scholar?q=P%2E+Koehn%2C+A%2E+Axelrod%2C+A%2E+B%2E+Mayne%2C+C%2E+Callison%2DBurch%2C+M%2E+Osborne%2C+D%2E+Talbot%2C+and+M%2E+White%2E+2005%2E+Edinburgh+system+description+for+the+2005+NIST+MT+evaluation%2E+In+Proceedings+of+Machine+Translation+Eva)
12. Philippe Langlais, Guihong Cao, and Fabrizio Gotti. 2005. RALI: SMT shared task system description. [find](https://scholar.google.com/scholar?q=Philippe+Langlais%2C+Guihong+Cao%2C+and+Fabrizio+Gotti%2E+2005%2E+RALI%3A+SMT+shared+task+system+description%2E)
13. In Proceedings of the 2nd ACL workshop on Building and Using Parallel Texts, pages 137–140, University of Michigan, Ann Arbor, June. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+2nd+ACL+workshop+on+Building+and+Using+Parallel+Texts%2C+pages+137%E2%80%93140%2C+University+of+Michigan%2C+Ann+Arbor%2C+June%2E)
14. Daniel Marcu and William Wong. 2002. [A phrase-based, joint probability model for statistical machine translation](https://md.research.elamur.ai/free/en/papers/marcu-2002-phrase-based-joint-probability-model/). In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP), Philadelphia, PA.
15. Robert C. Moore. 2004. Improving IBM word-alignment model 1. In Proceedings of the 42th Annual Meeting of the Association for Computational Linguistics (ACL), Barcelona, July. [find](https://scholar.google.com/scholar?q=Robert+C%2E+Moore%2E+2004%2E+Improving+IBM+word%2Dalignment+model+1%2E+In+Proceedings+of+the+42th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2C+Barcelona%2C+July%2E)
16. Hermann Ney, Ute Essen, and Reinhard Kneser. 1994. On structuring probabilistic dependencies in stochastic language modelling. Computer Speech and Language, 10:1–38. [find](https://scholar.google.com/scholar?q=Hermann+Ney%2C+Ute+Essen%2C+and+Reinhard+Kneser%2E+1994%2E+On+structuring+probabilistic+dependencies+in+stochastic+language+modelling%2E+Computer+Speech+and+Language%2C+10%3A1%E2%80%9338%2E)
17. Arthur Nádas. 1985. On Turing’s formula for word probabilities. IEEE Transactions on Acoustics, Speech and Signal Processing (ASSP), ASSP-33(6):1415–1417, December. [find](https://scholar.google.com/scholar?q=Arthur+Na%CC%81das%2E+1985%2E+On+Turing%E2%80%99s+formula+for+word+probabilities%2E+IEEE+Transactions+on+Acoustics%2C+Speech+and+Signal+Processing+%28ASSP%29%2C+ASSP%2D33%286%29%3A1415%E2%80%931417%2C+December%2E)
18. Franz Josef Och. 2003. Minimum error rate training for statistical machine translation. In Proceedings of the 41th Annual Meeting of the Association for Computational Linguistics (ACL), Sapporo, July. [find](https://scholar.google.com/scholar?q=Franz+Josef+Och%2E+2003%2E+Minimum+error+rate+training+for+statistical+machine+translation%2E+In+Proceedings+of+the+41th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2C+Sapporo%2C+July%2E)
19. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. [BLEU: A method for automatic evaluation of Machine Translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). Technical Report RC22176, IBM, September.
20. Andreas Stolcke. 2002. SRILM - an extensible language modeling toolkit. In Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP) 2002, Denver, Colorado, September. [find](https://scholar.google.com/scholar?q=Andreas+Stolcke%2E+2002%2E+SRILM+%2D+an+extensible+language+modeling+toolkit%2E+In+Proceedings+of+the+7th+International+Conference+on+Spoken+Language+Processing+%28ICSLP%29+2002%2C+Denver%2C+Colorado%2C+September%2E)
21. WMT. 2006. The NAACL Workshop on Statistical Machine Translation (www.statmt.org/wmt06), New York, June. [find](https://scholar.google.com/scholar?q=WMT%2E+2006%2E+The+NAACL+Workshop+on+Statistical+Machine+Translation+%28www%2Estatmt%2Eorg%2Fwmt06%29%2C+New+York%2C+June%2E)
22. Richard Zens and Hermann Ney. 2004. [Improvements in phrase-based statistical machine translation](https://md.research.elamur.ai/free/en/papers/zens-2004-improvements-phrase-based-statistical-machine/). In Proceedings of Human Language Technology Conference / North American Chapter of the ACL, Boston, May.

*Updated 24 October 2026*