---
title: "A Statistical Approach to Machine Translation — Brown 1990"
description: "The authors from IBM return to an old idea of Warren Weaver: machine translation can be done… 12 references, cited by 419 papers here."
authors: ["Peter F. Brown", "John Cocke", "Stephen A. Della Pietra", "Vincent J. Della Pietra", "Fredrick Jelinek", "John D. Lafferty", "Robert L. Mercer", "Paul S. Roossin"]
year: 1990
venue: "Computational Linguistics"
source: "https://aclanthology.org/J90-2002/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/brown-1990-statistical-approach-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/brown-1990-statistical-approach-machine-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1990](https://md.research.elamur.ai/free/en/papers/1990/) / A Statistical Approach to Machine Translation

# A Statistical Approach to Machine Translation

**Authors:** [Peter F. Brown](https://md.research.elamur.ai/free/en/authors/peter-f-brown/), [John Cocke](https://md.research.elamur.ai/free/en/authors/john-cocke/), [Stephen A. Della Pietra](https://md.research.elamur.ai/free/en/authors/stephen-a-della-pietra/), [Vincent J. Della Pietra](https://md.research.elamur.ai/free/en/authors/vincent-j-della-pietra/), Fredrick Jelinek, [John D. Lafferty](https://md.research.elamur.ai/free/en/authors/john-lafferty/), [Robert L. Mercer](https://md.research.elamur.ai/free/en/authors/robert-l-mercer/) and [Paul S. Roossin](https://md.research.elamur.ai/free/en/authors/paul-s-roossin/) · **Year:** 1990 · **Venue:** [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) ([Computational Linguistics 1990](https://md.research.elamur.ai/free/en/venues/cl/1990/)) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/J90-2002/) · [PDF](https://aclanthology.org/J90-2002.pdf) · [OpenAlex](https://openalex.org/W2097333193)

**Citations:** 1,655 citations (OpenAlex, 2026-10-14) · [419](https://md.research.elamur.ai/free/en/papers/brown-1990-statistical-approach-machine-translation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The authors from IBM return to an old idea of Warren Weaver: machine translation can be done statistically rather than with hand-written rules. They formulate the task as a search for the source sentence that most probably gave rise to the observed sentence. Bayes' theorem decomposes this task into three parts. The first is a source language model, here an n-gram model. The second is a translation model, in which each word has a fertility probability, i.e., how many words it generates, as well as lexical probabilities and probabilities of word-order distortion. The third is a decoder that searches for the best sentence using an inexact, suboptimal search. The parameters of both models are estimated automatically from the Canadian Hansard parallel corpus. The approach was tested on French-to-English translation in a task with a vocabulary of approximately 1,000 words. Only a small fraction of sentences matched the human translation exactly. Nevertheless, almost half of the outputs can be considered acceptable translations, and editing the system's output required noticeably fewer keystrokes than translating from scratch. This work laid the foundation for statistical machine translation (the IBM models), on which the systems of the following two decades relied.

## Key points
- **Noisy channel scheme.** One seeks the S for which the product Pr(S)·Pr(T|S) is largest. The language model is responsible for word order, and the translation model for word choice.
- **The "bag translation" experiment.** The words of an English sentence were shuffled, and their order was then restored with a trigram model. Of 38 sentences shorter than 11 words, 24 (63%) were restored exactly, and another 8 preserved the meaning, 84% in total.
- **Translation model parameters.** Fertility up to 25, lexical probabilities Pr(f|e), and a very simple distortion model Pr(i|j,l), where positions and lengths are limited to 25. Words without a counterpart are generated by a special word (null).
- **Test on 73 sentences from the Hansard.** Exact translations accounted for 5%, another 25% were synonymous phrasings, and 18% were different correct translations, so acceptable outputs totaled 48%. Ungrammatical outputs accounted for 37%. Correction took 776 keystrokes, whereas translation from scratch took 1,916, i.e., approximately 60% less work.
- **Limitations and plans.** The models were trained on only ~1% of the available data for translation and ~10% for the language model. The perplexity of the bigram model was ~78, while ~9 was expected for the trigram model. The plans include morphology, multi-word units such as "to go → aller", and syntactic grammars.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [IBM Research - Thomas J. Watson Research Center](https://md.research.elamur.ai/free/en/labs/ibm-research-thomas-j-watson-research-center/) — Research institute, Yorktown Heights, United States · 55 works · 49,277 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 1,105 works

💡 Concepts (1): [💡 Natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/) · 131 works

## Cite

Cite as: Peter F. Brown, John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer and Paul S. Roossin. 1990. A Statistical Approach to Machine Translation. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{brown1990statistical,
  title = {{A Statistical Approach to Machine Translation}},
  author = {Peter F. Brown, John Cocke, Stephen A. {Della Pietra}, Vincent J. {Della Pietra}, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer and Paul S. Roossin},
  year = {1990},
  doi = {10.5555/92858.92860},
  url = {https://doi.org/10.5555/92858.92860}
}
```

- [← previous Word Association Norms, Mutual Information, and Lexicography](https://md.research.elamur.ai/free/en/papers/church-1990-word-association-norms-mutual-information/)
- [next → Noun Classification From Predicate-Argument Structures](https://md.research.elamur.ai/free/en/papers/hindle-1990-noun-classification-predicate-argument-structures/)
- [↑ back to papers of 1990](https://md.research.elamur.ai/free/en/papers/1990/)

## References

12 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bahl, L. R.; Jelinek, F.; and Mercer, R. L. 1983 A Maximum Likelihood Approach to Continuous Speech Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-5(2): 179-190. [find](https://scholar.google.com/scholar?q=Bahl%2C+L%2E+R%2E%3B+Jelinek%2C+F%2E%3B+and+Mercer%2C+R%2E+L%2E+1983+A+Maximum+Likelihood+Approach+to+Continuous+Speech+Recognition%2E+IEEE+Transactions+on+Pattern+Analysis+and+Machine+Intelligence+PAMI%2D5%282%29%3A+179%2D190%2E)
2. Baker, J. K. 1979 Stochastic Modeling for Automatic Speech Understanding. In: Reddy, R. A. (ed.) Speech Recognition. Academic Press, New York, NY. [find](https://scholar.google.com/scholar?q=Baker%2C+J%2E+K%2E+1979+Stochastic+Modeling+for+Automatic+Speech+Understanding%2E+In%3A+Reddy%2C+R%2E+A%2E+%28ed%2E%29+Speech+Recognition%2E+Academic+Press%2C+New+York%2C+NY%2E)
3. Baum, L. E. 1972 An Inequality and Associated Maximization Technique in Statistical Estimation of Probabilistic Functions of a Markov Process. Inequalities 3:1-8. [find](https://scholar.google.com/scholar?q=Baum%2C+L%2E+E%2E+1972+An+Inequality+and+Associated+Maximization+Technique+in+Statistical+Estimation+of+Probabilistic+Functions+of+a+Markov+Process%2E+Inequalities+3%3A1%2D8%2E)
4. Bernstein, R. 1988 Howard's Way. The New York Times Magazine 138(47639): pp 40-44, 74, 92. [find](https://scholar.google.com/scholar?q=Bernstein%2C+R%2E+1988+Howard%27s+Way%2E+The+New+York+Times+Magazine+138%2847639%29%3A+pp+40%2D44%2C+74%2C+92%2E)
5. Dempster, A. P.; Laird, N. M.; and Rubin, D. B. 1977 Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society 39(B):1-38. [find](https://scholar.google.com/scholar?q=Dempster%2C+A%2E+P%2E%3B+Laird%2C+N%2E+M%2E%3B+and+Rubin%2C+D%2E+B%2E+1977+Maximum+Likelihood+from+Incomplete+Data+via+the+EM+Algorithm%2E+Journal+of+the+Royal+Statistical+Society+39%28B%29%3A1%2D38%2E)
6. Ferguson, J. D. 1980 Hidden Markov Analysis: An Introduction. In: Ferguson, J. D. (ed.), Hidden Markov Models for Speech. IDA-CRD, Princeton, NJ. [find](https://scholar.google.com/scholar?q=Ferguson%2C+J%2E+D%2E+1980+Hidden+Markov+Analysis%3A+An+Introduction%2E+In%3A+Ferguson%2C+J%2E+D%2E+%28ed%2E%29%2C+Hidden+Markov+Models+for+Speech%2E+IDA%2DCRD%2C+Princeton%2C+NJ%2E)
7. Garside, R. G.; Leech, G. N.; and Sampson, G. R. 1987 The Computational Analysis of English: A Corpus-Based Approach. Longman, NY. [find](https://scholar.google.com/scholar?q=Garside%2C+R%2E+G%2E%3B+Leech%2C+G%2E+N%2E%3B+and+Sampson%2C+G%2E+R%2E+1987+The+Computational+Analysis+of+English%3A+A+Corpus%2DBased+Approach%2E+Longman%2C+NY%2E)
8. Jelinck, F. and Mercer, R. L. 1980 Interpolated Estimation of Markov Source Parameters from Sparse Data. In: Proceedings of the Workshop on Pattern Recognition in Practice. North-Holland, Amsterdam, The Netherlands. [find](https://scholar.google.com/scholar?q=Jelinck%2C+F%2E+and+Mercer%2C+R%2E+L%2E+1980+Interpolated+Estimation+of+Markov+Source+Parameters+from+Sparse+Data%2E+In%3A+Proceedings+of+the+Workshop+on+Pattern+Recognition+in+Practice%2E+North%2DHolland%2C+Amsterdam%2C+T)
9. Sampson, G. R. 1986 A Stochastic Approach to Parsing. Proceedings of the I I th International Conference on Computational Linguistics. 151-155. [find](https://scholar.google.com/scholar?q=Sampson%2C+G%2E+R%2E+1986+A+Stochastic+Approach+to+Parsing%2E+Proceedings+of+the+I+I+th+International+Conference+on+Computational+Linguistics%2E+151%2D155%2E)
10. Sharman, R. A.; Jelinck, F.; and Mercer, R. L. 1988 Generating a Grammar for Statistical Training. In: Proceedings of the IBM Conference on Natural Language Processing, Thornwood, NY. [find](https://scholar.google.com/scholar?q=Sharman%2C+R%2E+A%2E%3B+Jelinck%2C+F%2E%3B+and+Mercer%2C+R%2E+L%2E+1988+Generating+a+Grammar+for+Statistical+Training%2E+In%3A+Proceedings+of+the+IBM+Conference+on+Natural+Language+Processing%2C+Thornwood%2C+NY%2E)
11. Sinclair, J. M. 1985 Lcxicographic Evidence. In: Ilson, R. (ed.) Dictionaries, Lexicography and Language Learning. Pergamon Press, N e w York, NY. [find](https://scholar.google.com/scholar?q=Sinclair%2C+J%2E+M%2E+1985+Lcxicographic+Evidence%2E+In%3A+Ilson%2C+R%2E+%28ed%2E%29+Dictionaries%2C+Lexicography+and+Language+Learning%2E+Pergamon+Press%2C+N+e+w+York%2C+NY%2E)
12. Weaver, W. 1955 Translation (1949). In: Machine Translation of Languages, M I T Press, Cambridge, M A . [find](https://scholar.google.com/scholar?q=Weaver%2C+W%2E+1955+Translation+%281949%29%2E+In%3A+Machine+Translation+of+Languages%2C+M+I+T+Press%2C+Cambridge%2C+M+A+%2E)

*Updated 1 November 2026*