--- title: "Learning Bilingual Lexicons from Monolingual — Haghighi 2008" description: "The paper addresses how to build a word-translation dictionary when no parallel text exists for a language pair, using only two separate monolingual…" authors: ["Aria Haghighi", "Percy Liang", "Taylor Berg-Kirkpatrick", "Dan Klein"] year: 2008 venue: "Proceedings of ACL-08: HLT" source: "https://aclanthology.org/P08-1088/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/haghighi-2008-learning-bilingual-lexicons-monolingual-corpora/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/haghighi-2008-learning-bilingual-lexicons-monolingual-corpora/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2008](https://md.research.elamur.ai/free/en/papers/2008/) / Learning Bilingual Lexicons from Monolingual Corpora # Learning Bilingual Lexicons from Monolingual Corpora **Authors:** [Aria Haghighi](https://md.research.elamur.ai/free/en/authors/aria-haghighi/), [Percy Liang](https://md.research.elamur.ai/free/en/authors/percy-liang/), [Taylor Berg-Kirkpatrick](https://md.research.elamur.ai/free/en/authors/taylor-berg-kirkpatrick/), [Dan Klein](https://md.research.elamur.ai/free/en/authors/dan-klein/) · **Year:** 2008 · **Venue:** Proceedings of ACL-08: HLT · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/P08-1088/) · [PDF](https://aclanthology.org/P08-1088.pdf) · [OpenAlex](https://openalex.org/W2140406733) **Citations:** 296 citations (openalex-snapshot · 2026-10-14) · [130](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper addresses how to build a word-translation dictionary when no parallel text exists for a language pair, using only two separate monolingual corpora and, optionally, a few seed translations. Each word type is represented by monolingual features: counts of surrounding context words and character substrings. The authors propose matching canonical correlation analysis (MCCA), a generative model in which translation pairs share a latent, language-independent concept vector projected into each language's feature space. Words without a translation are generated from a separate baseline distribution. Learning uses a hard-EM loop. The E-step finds the best bipartite matching with the Hungarian algorithm, and the M-step fits probabilistic CCA to the current pairs. The number of accepted pairs grows gradually over iterations. Experiments cover English paired with Spanish, French, Chinese, Arabic and German, using Wikipedia, Europarl, Gigaword, Xinhua and UN data. On topically similar but non-parallel English-Spanish text, the best setup reaches 89.0% precision at 33% recall. This shows that useful lexicons can be mined for resource-poor pairs. ## Key points - **Corpora.** EN-ES-W uses 3,851 Wikipedia articles that exist in both languages. EN-ES(FR)-D, EN-CH-D and EN-AR-D take the first 50k sentences of a parallel corpus for English and the second 50k for the other language, so no sentence is shared. EN-ES-G uses 100k sentences each of English and Spanish Gigaword. Alignments are never used. - **Evaluation setup.** Only nouns are scored, following Koehn and Knight (2002). A word counts as a noun if its most frequent tag is a noun. - **Languages without useful spelling overlap.** With context features alone, English-Chinese and English-Arabic reach 26.8 and 31.0 precision at 0.33 recall. At 0.1 recall they reach 60.1 and 70.0. - **Comparison with prior work.** In a setup approximating Koehn and Knight (English Gigaword vs. German Europarl), MCCA scores 61.7% accuracy on its 186 most confident outputs. Koehn and Knight reported 39%. - **Error analysis on EN-ES-W.** Of the top 100 "errors": - 21 were valid translations missing from Wiktionary; - 30 were semantically related words; - 15 were orthographic look-alikes; - 4 were morphological variants; - 30 were uncategorized. Adding an alignment-derived lexicon to the reference raised precision at 0.33 recall from 92.3 to 98.0. ## Interesting ideas - **Alternate between matching and projection fitting.** The loop switches between solving a one-to-one assignment problem and refitting a shared projection space. This hard-EM pattern could carry over to other unsupervised alignment tasks, such as linking entities, place names or product categories across two languages. - **Grow the trusted set slowly.** Only the highest-scoring pairs are kept at first, and the number of retained edges rises with each iteration. This keeps early noise from corrupting the model and is a cheap safeguard for any self-training loop. - **Use a cheap proxy score.** The exact likelihood-based edge weight was replaced by a constant minus the squared distance between the words' expected latent vectors. This was faster and even slightly more accurate, a reminder that a simple geometric score can beat a principled but costly one. - **Simulate a non-parallel setting from parallel data.** Different halves of a parallel corpus serve as the two monolingual corpora. This gives realistic comparable data while the domain stays controlled, which helps when building benchmarks for low-resource pairs. - **Audit the reference dictionary itself.** Manually categorizing the top errors revealed many correct pairs absent from Wiktionary, and a supplementary lexicon changed the measured precision considerably. Projecting features into the shared space also gave interpretable correspondences, such as suffix mappings. Both checks are worth repeating whenever a gold lexicon may be incomplete. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 130 papers of our library cite it (of the 140884 whose bibliographies we read); 12 of them have a page here, 118 not yet. - [Neural Decipherment via Minimum-Cost Flow: From Ugaritic to Linear B](https://md.research.elamur.ai/free/en/papers/acl-2019-luo-neural-decipherment-minimum-cost-flow/) - [Massively Multilingual Word Embeddings](https://md.research.elamur.ai/free/en/papers/ammar-2016-massively-multilingual-word-embeddings/) - [Improving Vector Space Word Representations Using Multilingual Correlation](https://md.research.elamur.ai/free/en/papers/faruqui-2014-improving-vector-space-word-representations/) - [Cross-lingual Dependency Parsing Based on Distributed Representations](https://md.research.elamur.ai/free/en/papers/guo-2015-cross-lingual-dependency-parsing-based/) - [Multilingual Models for Compositional Distributed Semantics](https://md.research.elamur.ai/free/en/papers/hermann-2014-multilingual-models-compositional-distributed-semantics/) - [SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation](https://md.research.elamur.ai/free/en/papers/hill-2015-simlex-999-evaluating-semantic-models/) - [Bilingual Word Representations with Monolingual Quality in Mind](https://md.research.elamur.ai/free/en/papers/luong-2015-bilingual-word-representations-monolingual-quality/) - [Extracting Parallel Sentences from Comparable Corpora using Document Level Alignment](https://md.research.elamur.ai/free/en/papers/smith-2010-extracting-parallel-sentences-comparable-corpora/) - [Offline bilingual word vectors, orthogonal transformations and the inverted softmax](https://md.research.elamur.ai/free/en/papers/smith-2017-offline-bilingual-word-vectors-orthogonal/) - [On the Limitations of Unsupervised Bilingual Dictionary Induction](https://md.research.elamur.ai/free/en/papers/sogaard-2018-limitations-unsupervised-bilingual-dictionary-induction/) - [Crowdsourcing Translation: Professional Quality from Non-Professionals](https://md.research.elamur.ai/free/en/papers/zaidan-2011-crowdsourcing-translation-professional-quality-non/) - [Adversarial Training for Unsupervised Bilingual Lexicon Induction](https://md.research.elamur.ai/free/en/papers/zhang-2017-adversarial-training-unsupervised-bilingual-lexicon/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{haghighi2008learning, title = {{Learning Bilingual Lexicons from Monolingual Corpora}}, author = {Haghighi, Aria and Liang, Percy and Berg-Kirkpatrick, Taylor and Klein, Dan}, booktitle = {Proceedings of ACL-08: HLT}, year = {2008}, url = {https://aclanthology.org/P08-1088/} } ``` - [← previous Lattice-based Minimum Error Rate Training for Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/macherey-2008-lattice-based-minimum-error-rate/) - [next → Learning Morphology with Morfette](https://md.research.elamur.ai/free/en/papers/lingbulk-2008-chrupala-learning-morphology-morfette/) - [↑ back to papers of 2008](https://md.research.elamur.ai/free/en/papers/2008/) ## References 16 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Francis R. Bach and Michael I. Jordan. 2006. A probabilistic interpretation of canonical correlation analysis. Technical report, University of California, Berkeley. [find](https://scholar.google.com/scholar?q=Francis+R%2E+Bach+and+Michael+I%2E+Jordan%2E+2006%2E+A+probabilistic+interpretation+of+canonical+correlation+analysis%2E+Technical+report%2C+University+of+California%2C+Berkeley%2E) 2. Peter F. Brown, Stephen Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1994. The mathematic of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–311. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Stephen+Della+Pietra%2C+Vincent+J%2E+Della+Pietra%2C+and+Robert+L%2E+Mercer%2E+1994%2E+The+mathematic+of+statistical+machine+translation%3A+Parameter+estimation%2E+Computational+Linguistics%2C+19%282%29%3A263) 3. Mona Diab, Kadri Hacioglu, and Daniel Jurafsky. 2004. Automatic tagging of arabic text: From raw text to base phrase chunks. In HLT-NAACL. [find](https://scholar.google.com/scholar?q=Mona+Diab%2C+Kadri+Hacioglu%2C+and+Daniel+Jurafsky%2E+2004%2E+Automatic+tagging+of+arabic+text%3A+From+raw+text+to+base+phrase+chunks%2E+In+HLT%2DNAACL%2E) 4. Pascale Fung. 1995. Compiling bilingual lexicon entries from a non-parallel english-chinese corpus. In Third Annual Workshop on Very Large Corpora. [find](https://scholar.google.com/scholar?q=Pascale+Fung%2E+1995%2E+Compiling+bilingual+lexicon+entries+from+a+non%2Dparallel+english%2Dchinese+corpus%2E+In+Third+Annual+Workshop+on+Very+Large+Corpora%2E) 5. Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang, and Ignacio Thayer. 2006. Scalable inference and training of context-rich syntactic translation models. In COLING-ACL. [find](https://scholar.google.com/scholar?q=Michel+Galley%2C+Jonathan+Graehl%2C+Kevin+Knight%2C+Daniel+Marcu%2C+Steve+DeNeefe%2C+Wei+Wang%2C+and+Ignacio+Thayer%2E+2006%2E+Scalable+inference+and+training+of+context%2Drich+syntactic+translation+models%2E+In+COLING%2DA) [our card](https://md.research.elamur.ai/free/en/papers/galley-2006-scalable-inference-training-context-rich/) 6. David R. Hardoon, Sandor Szedmak, and John Shawe-Taylor. 2003. Canonical correlation analysis an overview with application to learning methods. Technical Report CSD-TR-03-02, Royal Holloway University of London. [find](https://scholar.google.com/scholar?q=David+R%2E+Hardoon%2C+Sandor+Szedmak%2C+and+John+Shawe%2DTaylor%2E+2003%2E+Canonical+correlation+analysis+an+overview+with+application+to+learning+methods%2E+Technical+Report+CSD%2DTR%2D03%2D02%2C+Royal+Holloway+University) 7. Philipp Koehn and Kevin Knight. 2002. Learning a translation lexicon from monolingual corpora. In Proceedings of ACL Workshop on Unsupervised Lexical Acquisition. [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Kevin+Knight%2E+2002%2E+Learning+a+translation+lexicon+from+monolingual+corpora%2E+In+Proceedings+of+ACL+Workshop+on+Unsupervised+Lexical+Acquisition%2E) [our card](https://md.research.elamur.ai/free/en/papers/koehn-2002-learning-translation-lexicon-monolingual-corpora/) 8. P. Koehn. 2004. Pharaoh: A beam search decoder for phrase-based statistical machine translation models. In Proceedings of AMTA 2004. [find](https://scholar.google.com/scholar?q=P%2E+Koehn%2E+2004%2E+Pharaoh%3A+A+beam+search+decoder+for+phrase%2Dbased+statistical+machine+translation+models%2E+In+Proceedings+of+AMTA+2004%2E) 9. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In MT Summit. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2005%2E+Europarl%3A+A+parallel+corpus+for+statistical+machine+translation%2E+In+MT+Summit%2E) 10. H. W. Kuhn. 1955. The Hungarian method for the assignment problem. Naval Research Logistic Quarterly. [find](https://scholar.google.com/scholar?q=H%2E+W%2E+Kuhn%2E+1955%2E+The+Hungarian+method+for+the+assignment+problem%2E+Naval+Research+Logistic+Quarterly%2E) 11. P. Liang, D. Klein, and M. I. Jordan. 2008. Agreement-based learning. In NIPS. [find](https://scholar.google.com/scholar?q=P%2E+Liang%2C+D%2E+Klein%2C+and+M%2E+I%2E+Jordan%2E+2008%2E+Agreement%2Dbased+learning%2E+In+NIPS%2E) 12. Reinhard Rapp. 1995. Identifying word translation in non-parallel texts. In ACL. [find](https://scholar.google.com/scholar?q=Reinhard+Rapp%2E+1995%2E+Identifying+word+translation+in+non%2Dparallel+texts%2E+In+ACL%2E) 13. Helmut Schmid. 1994. Probabilistic part-of-speech tagging using decision trees. In International Conference on New Methods in Language Processing. [find](https://scholar.google.com/scholar?q=Helmut+Schmid%2E+1994%2E+Probabilistic+part%2Dof%2Dspeech+tagging+using+decision+trees%2E+In+International+Conference+on+New+Methods+in+Language+Processing%2E) 14. N. Smith and J. Eisner. 2006. Annealing structural bias in multilingual weighted grammar induction. In ACL. [find](https://scholar.google.com/scholar?q=N%2E+Smith+and+J%2E+Eisner%2E+2006%2E+Annealing+structural+bias+in+multilingual+weighted+grammar+induction%2E+In+ACL%2E) 15. L. G. Valiant. 1979. The complexity of computing the permanent. Theoretical Computer Science, 8:189– 201. [find](https://scholar.google.com/scholar?q=L%2E+G%2E+Valiant%2E+1979%2E+The+complexity+of+computing+the+permanent%2E+Theoretical+Computer+Science%2C+8%3A189%E2%80%93+201%2E) 16. D. Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In ACL. [find](https://scholar.google.com/scholar?q=D%2E+Yarowsky%2E+1995%2E+Unsupervised+word+sense+disambiguation+rivaling+supervised+methods%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/yarowsky-1995-unsupervised-word-sense-disambiguation-rivaling/)