--- title: "Learning a Translation Lexicon from Monolingual — Koehn 2002" description: "Statistical MT usually learns word translations from parallel text." authors: ["Philipp Koehn", "Kevin Knight"] year: 2002 venue: "Proceedings of the ACL-02 Workshop on Unsupervised Lexical Acquisition" source: "https://aclanthology.org/W02-0902/" doi: "10.3115/1118627.1118629" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/koehn-2002-learning-translation-lexicon-monolingual-corpora/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/koehn-2002-learning-translation-lexicon-monolingual-corpora/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2002](https://md.research.elamur.ai/free/en/papers/2002/) / Learning a Translation Lexicon from Monolingual Corpora # Learning a Translation Lexicon from Monolingual Corpora **Authors:** [Philipp Koehn](https://md.research.elamur.ai/free/en/authors/philipp-koehn/), [Kevin Knight](https://md.research.elamur.ai/free/en/authors/kevin-knight/) · **Institutions:** [University of Southern California](https://md.research.elamur.ai/free/en/universities/university-of-southern-california/) · **Year:** 2002 · **Venue:** Proceedings of the ACL-02 Workshop on Unsupervised Lexical Acquisition · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W02-0902/) · [PDF](https://aclanthology.org/W02-0902.pdf) · [DOI 10.3115/1118627.1118629](https://doi.org/10.3115/1118627.1118629) · [OpenAlex](https://openalex.org/W2121415745) **Citations:** 224 citations (OpenAlex, 2026-10-14) · [108](https://md.research.elamur.ai/free/en/papers/koehn-2002-learning-translation-lexicon-monolingual-corpora/cited-by/) in our library, of the 140884 papers whose bibliographies we read ## Summary Statistical MT usually learns word translations from parallel text. Such text is scarce for most language pairs and often limited to parliamentary or legal domains, while monolingual text is plentiful. Koehn and Knight test how far one can get building a one-to-one German–English noun lexicon from two unrelated monolingual news collections: the 1990–1992 Wall Street Journal and the 1995–1996 German DPA newswire. They first build a seed of 1,339 pairs. These are words spelled the same in both languages, plus words that match after a few simple letter-rewriting rules. They then test four further clues: spelling similarity, shared context, preserved similarity between words, and frequency. Each clue scores candidate pairs, and a greedy one-to-one matching over 1,000 frequent nouns picks the final entries. Spelling and context each add over a hundred correct entries, and summing their scores works best. Accuracy against a parallel test corpus rises from 15.8% with the seed alone to about 39%. The work matters because it builds a usable bootstrap lexicon with no bilingual data, which helps low-resource languages and new domains. ## Key points - **Seed lexicon from identical spellings** - 976 German nouns have an English word with exactly the same spelling, and 88% of these are real translations. - Reliability grows with word length: about 60% for 3-letter words versus 98% for 10-letter words. - Keeping only words of 6 or more letters gives 622 pairs at 96% accuracy. - **Rewriting rules** - Two German-to-English rules (k/z → c, and -tät → -ty) add 363 pairs, of which 330 (91%) are correct. - Together with the identical words, this gives the 1,339-pair seed. - **Spelling and context clues** - Spelling similarity uses the longest common subsequence ratio (Melamed, 1995). - Context vectors follow Rapp (1999), but use a four-noun window that skips all non-noun words in between. - The context vectors are translated through the seed pairs so they can be compared across languages. - **Results on 1,000 frequent nouns** - Spelling alone adds 140 correct entries (25.4% corpus score), and context alone adds 107 (31.9%). - The similarity and frequency clues add only 2 entries each. - Spelling+context gives 185 entries (38.6%), and all clues together give 186 (39.0%). - A random baseline gets about 1 correct entry, while a lexicon learned from a large parallel corpus reaches 89%. - **Cost** - The greedy matching is quadratic in vocabulary size and took up to 3 days for 1,000 words. - The authors estimate a run on 10,000 words would take nearly a year. ## Interesting ideas - **Free seed dictionary from shared strings.** Identically spelled words, filtered by length, give a cheap and fairly precise seed. This is attractive for any language pair that shares loanwords, international terms or names; in a Cyrillic–Latin setting it would first need transliteration. - **Breaking the chicken-and-egg loop.** Context vectors normally need a dictionary to translate them. Here the seed pairs serve as the "translatable" context dimensions, and this bootstrapping pattern can be reused for other cross-lingual mappings. - **Flooring weak similarity scores.** All spelling scores below 0.3 are treated as 0.3, because such small overlaps are accidental. This stops noise from dominating when several signals are added, a trick worth applying in other score-fusion setups. - **Greedy one-to-one assignment.** The highest-scoring pair is fixed, both words are removed, and the process repeats. This is a simple, practical alternative to exhaustive matching, though it is still costly at large vocabulary sizes. - **Two complementary evaluation views.** The paper counts correct dictionary entries and also measures token-weighted accuracy against a parallel corpus. This shows which clue helps on frequent words (context) versus on many rarer types (spelling). *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{koehn2002learning, title = {{Learning a Translation Lexicon from Monolingual Corpora}}, author = {Koehn, Philipp and Knight, Kevin}, booktitle = {Proceedings of the ACL-02 Workshop on Unsupervised Lexical Acquisition}, year = {2002}, doi = {10.3115/1118627.1118629}, url = {https://doi.org/10.3115/1118627.1118629} } ``` - [← previous A Generative Constituent-Context Model for Improved Grammar Induction](https://md.research.elamur.ai/free/en/papers/klein-2002-generative-constituent-context-model-improved/) - [next → Toward a Broad-coverage Bilingual Corpus for Speech Translation of Travel Conversations in the Real World](https://md.research.elamur.ai/free/en/papers/takezawa-2002-broad-coverage-bilingual-corpus-speech/) - [↑ back to papers of 2002](https://md.research.elamur.ai/free/en/papers/2002/) ## References 13 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Al-Onaizan, Y., Curin, J., Jahr, M., Knight, K., Lafferty, J., Melamed, D., Och, F.-J., Purdy, D., Smith, N. A., and Yarowsky, D. (1999). Statistical machine translation. Technical report, John Hopkins University Summer Workshop [http://www.clsp.jhu.edu/ws99/projects/mt/](http://www.clsp.jhu.edu/ws99/projects/mt/). [↗](http://www.clsp.jhu.edu/ws99/projects/mt/) 2. Brown, P., Cocke, J., Pietra, S. A. D., Pietra, V. J. D., Jelinek, F., Lafferty, J. D., Mercer, R. L., and Rossin, P. (1990). A statistical approach to machine translation. Computational Linguistics, 16(2):76–85. [find](https://scholar.google.com/scholar?q=Brown%2C+P%2E%2C+Cocke%2C+J%2E%2C+Pietra%2C+S%2E+A%2E+D%2E%2C+Pietra%2C+V%2E+J%2E+D%2E%2C+Jelinek%2C+F%2E%2C+Lafferty%2C+J%2E+D%2E%2C+Mercer%2C+R%2E+L%2E%2C+and+Rossin%2C+P%2E+%281990%29%2E+A+statistical+approach+to+machine+translation%2E+Computational+Linguistics%2C+) 3. Diab, M. and Finch, S. (2000). A statistical word-level translation model for comparable corpora. In Proceedings of the Conference on Content-based multimedia information access (RIAO). [find](https://scholar.google.com/scholar?q=Diab%2C+M%2E+and+Finch%2C+S%2E+%282000%29%2E+A+statistical+word%2Dlevel+translation+model+for+comparable+corpora%2E+In+Proceedings+of+the+Conference+on+Content%2Dbased+multimedia+information+access+%28RIAO%29%2E) 4. Fung, P. and Yee, L. Y. (1998). An IR approach for translating new words from nonparallel, comparable texts. In Proceedings of ACL 36, pages 414–420. [find](https://scholar.google.com/scholar?q=Fung%2C+P%2E+and+Yee%2C+L%2E+Y%2E+%281998%29%2E+An+IR+approach+for+translating+new+words+from+nonparallel%2C+comparable+texts%2E+In+Proceedings+of+ACL+36%2C+pages+414%E2%80%93420%2E) [our card](https://md.research.elamur.ai/free/en/papers/fung-1998-ir-approach-translating-new-words/) 5. Jones, K. S. (1979). Experiments in relevance weighting of search terms. In Information Processing and Management, pages 133–144. [find](https://scholar.google.com/scholar?q=Jones%2C+K%2E+S%2E+%281979%29%2E+Experiments+in+relevance+weighting+of+search+terms%2E+In+Information+Processing+and+Management%2C+pages+133%E2%80%93144%2E) 6. Koehn, P. (2002). Combining multiclass maximum entropy classifiers with neural network voting. In Proceedings of PorTAL. [find](https://scholar.google.com/scholar?q=Koehn%2C+P%2E+%282002%29%2E+Combining+multiclass+maximum+entropy+classifiers+with+neural+network+voting%2E+In+Proceedings+of+PorTAL%2E) 7. Koehn, P. and Knight, K. (2000). Estimating word translation probabilities from unrelated monolingual corpora using the EM algorithm. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Koehn%2C+P%2E+and+Knight%2C+K%2E+%282000%29%2E+Estimating+word+translation+probabilities+from+unrelated+monolingual+corpora+using+the+EM+algorithm%2E+In+Proceedings+of+AAAI%2E) 8. Koehn, P. and Knight, K. (2001). Knowledge sources for word-level translation models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP. [find](https://scholar.google.com/scholar?q=Koehn%2C+P%2E+and+Knight%2C+K%2E+%282001%29%2E+Knowledge+sources+for+word%2Dlevel+translation+models%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+EMNLP%2E) 9. Mann, G. S. and Yarowski, D. (2001). Multi-path translation lexicon induction via bridge languages. In Proceedings of NAACL, pages 151–158. [find](https://scholar.google.com/scholar?q=Mann%2C+G%2E+S%2E+and+Yarowski%2C+D%2E+%282001%29%2E+Multi%2Dpath+translation+lexicon+induction+via+bridge+languages%2E+In+Proceedings+of+NAACL%2C+pages+151%E2%80%93158%2E) 10. Melamed, D. (1995). Automatic evaluation and uniform filter cascades for inducing n-best translation lexicons. In Third Workshop on Very Large Corpora. [find](https://scholar.google.com/scholar?q=Melamed%2C+D%2E+%281995%29%2E+Automatic+evaluation+and+uniform+filter+cascades+for+inducing+n%2Dbest+translation+lexicons%2E+In+Third+Workshop+on+Very+Large+Corpora%2E) 11. Rapp, R. (1995). Identifying word translations in non-parallel texts. In Proceedings of ACL 33, pages 320–322. [find](https://scholar.google.com/scholar?q=Rapp%2C+R%2E+%281995%29%2E+Identifying+word+translations+in+non%2Dparallel+texts%2E+In+Proceedings+of+ACL+33%2C+pages+320%E2%80%93322%2E) [our card](https://md.research.elamur.ai/free/en/papers/rapp-1995-identifying-word-translations-non-parallel/) 12. Rapp, R. (1999). Automatic identification of word translations from unrelated English and German corpora. In Proceedings of ACL 37, pages 519–526. [find](https://scholar.google.com/scholar?q=Rapp%2C+R%2E+%281999%29%2E+Automatic+identification+of+word+translations+from+unrelated+English+and+German+corpora%2E+In+Proceedings+of+ACL+37%2C+pages+519%E2%80%93526%2E) [our card](https://md.research.elamur.ai/free/en/papers/rapp-1999-automatic-identification-word-translations-unrelated/) 13. Tiedemann, J. (1999). Automatic construction of weighted string similarity measures. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP. [find](https://scholar.google.com/scholar?q=Tiedemann%2C+J%2E+%281999%29%2E+Automatic+construction+of+weighted+string+similarity+measures%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+EMNLP%2E)