---
title: "Bilingual Lexicon Induction with Semi-supervision in"
description: "They propose BLISS, a semi-supervised framework that jointly optimises three objectives: Supervised… 26 references, cited by 58 papers here."
authors: ["Barun Patra", "Joel Ruben Antony Moniz", "Sarthak Garg", "Matthew R. Gormley", "Graham Neubig"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1018/"
doi: "10.18653/v1/p19-1018"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/patra-2019-bilingual-lexicon-induction-semi-supervision/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/patra-2019-bilingual-lexicon-induction-semi-supervision/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces

# Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces

**Authors:** [Barun Patra](https://md.research.elamur.ai/free/en/authors/barun-patra/), [Joel Ruben Antony Moniz](https://md.research.elamur.ai/free/en/authors/joel-ruben-antony-moniz/), [Sarthak Garg](https://md.research.elamur.ai/free/en/authors/sarthak-garg/), [Matthew R. Gormley](https://md.research.elamur.ai/free/en/authors/matthew-r-gormley/) and [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/) · **Year:** 2019 · **Venue:** [Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2019](https://md.research.elamur.ai/free/en/venues/acl/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1018/) · [PDF](https://aclanthology.org/P19-1018.pdf) · [DOI 10.18653/v1/p19-1018](https://doi.org/10.18653/v1/p19-1018) · [OpenAlex](https://openalex.org/W2950797315)

**Citations:** 100 citations (OpenAlex, 2026-10-14) · [58](https://md.research.elamur.ai/free/en/papers/patra-2019-bilingual-lexicon-induction-semi-supervision/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Most bilingual lexicon induction (BLI) methods map one monolingual embedding space onto another with an orthogonal matrix. This only works if the two spaces have roughly the same shape (isometry). The authors test that assumption by approximating the Gromov-Hausdorff distance between embedding spaces. They find it holds poorly for language pairs that are etymologically or typologically far apart.

They propose BLISS, a semi-supervised framework that jointly optimises three objectives:
- **Supervised loss:** aligns a small seed dictionary.
- **Adversarial loss:** matches the distributions of the full unaligned vocabularies.
- **Soft orthogonality loss:** a back-translation (cyclic) consistency term that encourages, but does not force, an orthogonal mapping.

Iterative Procrustes refinement with a new hubness filter further improves the mapping.

On the MUSE benchmark, BLISS sets new state-of-the-art results on most language pairs. Its gains are largest where the spaces look non-isometric. Even a small amount of supervision makes training much more stable. This matters for distant or low-resource pairs, where purely unsupervised methods tend to fail.

## Key points
- **Measuring isometry:** The Gromov-Hausdorff distance is approximated by the Bottleneck distance, following Chazal et al. (2009). Values are higher for distant language pairs.
- **Training losses:** BLISS's adversarial loss resembles Lample et al. (2018) and uses an MLP discriminator. Its supervised loss covers seed pairs. Its weak orthogonality term compares x with WᵀWx.
- **Hubness filtering:** During iterative dictionary expansion, target words that are nearest neighbours of too many source words are excluded. This gives a small accuracy gain.
- **Reported results:** The abstract claims state-of-the-art results on 15 of 18 MUSE pairs. The introduction instead says the best model wins 10 of 16 MUSE pairs. The supplied text does not resolve this inconsistency, so check the full paper before citing.
- **Stability and errors:**
  - In a toy experiment, unsupervised matching found the correct rotation only about half the time.
  - In English–Russian, the unsupervised baseline lands on the right semantic family but picks the wrong word (aunt → бабушка instead of тетя).
  - Remaining errors are mostly caused by polysemy and antonymy.
  - Code is available at github.com/joelmoniz/BLISS.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 365 works · 158,048 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Barun Patra, Joel Ruben Antony Moniz, Sarthak Garg, Matthew R. Gormley and Graham Neubig. 2019. Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{patra2019bilingual,
  title = {{Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces}},
  author = {Barun Patra, Joel Ruben Antony Moniz, Sarthak Garg, Matthew R. Gormley and Graham Neubig},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1018},
  url = {https://doi.org/10.18653/v1/p19-1018}
}
```

- [← previous Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation](https://md.research.elamur.ai/free/en/papers/karpukhin-2019-training-synthetic-noise-improves-robustness/)
- [next → Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?](https://md.research.elamur.ai/free/en/papers/vulic-2019-do-we-really-need-fully/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

26 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aria Haghighi, Percy Liang, Taylor Berg-Kirkpatrick, and Dan Klein. 2008. [Learning bilingual lexicons from monolingual corpora](https://md.research.elamur.ai/free/en/papers/haghighi-2008-learning-bilingual-lexicons-monolingual-corpora/). In Proceedings of ACL-08: HLT, pages 771–779, Columbus, Ohio. Association for Computational Linguistics.
2. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2016. [Learning principled bilingual mappings of word embeddings while preserving monolingual invariance](https://md.research.elamur.ai/free/en/papers/artetxe-2016-learning-principled-bilingual-mappings-word/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2289–2294.
3. Ann Irvine and Chris Callison-Burch. 2013. Combining bilingual and comparable corpora for low resource machine translation. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 262–270, Sofia, Bulgaria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ann+Irvine+and+Chris+Callison%2DBurch%2E+2013%2E+Combining+bilingual+and+comparable+corpora+for+low+resource+machine+translation%2E+In+Proceedings+of+the+Eighth+Workshop+on+Statistical+Machine+Translation%2C+pa)
4. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2017. [Learning bilingual word embeddings with (almost) no bilingual data](https://md.research.elamur.ai/free/en/papers/artetxe-2017-learning-bilingual-word-embeddings-almost/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 451–462, Vancouver, Canada. Association for Computational…
5. Pratik Jawanpuria, Arjun Balgovind, Anoop Kunchukuttan, and Bamdev Mishra. 2018. Learning multilingual word embeddings in latent metric space: a geometric approach. arXiv preprint arXiv:1808.08773. [find](https://scholar.google.com/scholar?q=Pratik+Jawanpuria%2C+Arjun+Balgovind%2C+Anoop+Kunchukuttan%2C+and+Bamdev+Mishra%2E+2018%2E+Learning+multilingual+word+embeddings+in+latent+metric+space%3A+a+geometric+approach%2E+arXiv+preprint+arXiv%3A1808%2E08773%2E)
6. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018a. Generalizing and improving bilingual word embedding mappings with a multi-step framework of linear transformations. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18). [find](https://scholar.google.com/scholar?q=Mikel+Artetxe%2C+Gorka+Labaka%2C+and+Eneko+Agirre%2E+2018a%2E+Generalizing+and+improving+bilingual+word+embedding+mappings+with+a+multi%2Dstep+framework+of+linear+transformations%2E+In+Proceedings+of+the+Thirty%2DS)
7. Armand Joulin, Piotr Bojanowski, Tomas Mikolov, Hervé Jégou, and Edouard Grave. 2018. [Loss in translation: Learning bilingual word mapping with a retrieval criterion](https://md.research.elamur.ai/free/en/papers/joulin-2018-loss-translation-learning-bilingual-word/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2979–2984.
8. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018b. [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://md.research.elamur.ai/free/en/papers/artetxe-2018-vecmap-unsupervised/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 789–798.
9. Yova Kementchedjhieva, Sebastian Ruder, Ryan Cotterell, and Anders Søgaard. 2018. Generalizing procrustes analysis for better bilingual dictionary induction. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 211–220. [find](https://scholar.google.com/scholar?q=Yova+Kementchedjhieva%2C+Sebastian+Ruder%2C+Ryan+Cotterell%2C+and+Anders+S%C3%B8gaard%2E+2018%2E+Generalizing+procrustes+analysis+for+better+bilingual+dictionary+induction%2E+In+Proceedings+of+the+22nd+Conference+on+C)
10. Min Xiao and Yuhong Guo. 2014. Distributed word representation learning for cross-lingual dependency parsing. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, pages 119–129. [find](https://scholar.google.com/scholar?q=Min+Xiao+and+Yuhong+Guo%2E+2014%2E+Distributed+word+representation+learning+for+cross%2Dlingual+dependency+parsing%2E+In+Proceedings+of+the+Eighteenth+Conference+on+Computational+Natural+Language+Learning%2C+pa)
11. Alexandre Klementiev, Ivan Titov, and Binod Bhattarai. 2012. [Inducing crosslingual distributed representations of words](https://md.research.elamur.ai/free/en/papers/klementiev-2012-inducing-crosslingual-distributed-representations-words/). pages 1459–1474.
12. Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. 2015. [Normalized word embedding and orthogonal transform for bilingual word translation](https://md.research.elamur.ai/free/en/papers/xing-2015-normalized-word-embedding-orthogonal-transform/). In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1006–1011.
13. Guillaume Lample, Alexis Conneau, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018. Word translation without parallel data. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Alexis+Conneau%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Ludovic+Denoyer%2C+and+Herve%CC%81+Je%CC%81gou%2E+2018%2E+Word+translation+without+parallel+data%2E+In+International+Conference+on+Learning+Representations%2E)
14. Thang Luong, Hieu Pham, and Christopher D Manning. 2015. [Bilingual word representations with monolingual quality in mind](https://md.research.elamur.ai/free/en/papers/luong-2015-bilingual-word-representations-monolingual-quality/). In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pages 151–159.
15. Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017a. [Adversarial training for unsupervised bilingual lexicon induction](https://md.research.elamur.ai/free/en/papers/zhang-2017-adversarial-training-unsupervised-bilingual-lexicon/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1959–1970.
16. Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013a. Exploiting similarities among languages for machine translation. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Quoc+V+Le%2C+and+Ilya+Sutskever%2E+2013a%2E+Exploiting+similarities+among+languages+for+machine+translation%2E)
17. Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017b. Earth mover’s distance minimization for unsupervised bilingual lexicon induction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1934–1945. [find](https://scholar.google.com/scholar?q=Meng+Zhang%2C+Yang+Liu%2C+Huanbo+Luan%2C+and+Maosong+Sun%2E+2017b%2E+Earth+mover%E2%80%99s+distance+minimization+for+unsupervised+bilingual+lexicon+induction%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+)
18. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, p…
19. Yuan Zhang, David Gaddy, Regina Barzilay, and Tommi Jaakkola. 2016. Ten pairs to tag-multilingual pos tagging via coarse mapping between embeddings. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yuan+Zhang%2C+David+Gaddy%2C+Regina+Barzilay%2C+and+Tommi+Jaakkola%2E+2016%2E+Ten+pairs+to+tag%2Dmultilingual+pos+tagging+via+coarse+mapping+between+embeddings%2E+Association+for+Computational+Linguistics%2E)
20. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
21. Will Y Zou, Richard Socher, Daniel Cer, and Christopher D Manning. 2013. [Bilingual word embeddings for phrase-based machine translation](https://md.research.elamur.ai/free/en/papers/zou-2013-bilingual-word-embeddings-phrase-based/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1393–1398.
22. Ye Qi, Devendra Sachan, Matthieu Felix, Sarguna Padmanabhan, and Graham Neubig. 2018. [When and why are pre-trained word embeddings useful for neural machine translation?](https://md.research.elamur.ai/free/en/papers/qi-2018-when-why-are-pre-trained/) In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua…
23. Miloš Radovanović, Alexandros Nanopoulos, and Mirjana Ivanović. 2010. Hubs in space: Popular nearest neighbors in high-dimensional data. volume 11, pages 2487–2531. [find](https://scholar.google.com/scholar?q=Milos%CC%8C+Radovanovic%CC%81%2C+Alexandros+Nanopoulos%2C+and+Mirjana+Ivanovic%CC%81%2E+2010%2E+Hubs+in+space%3A+Popular+nearest+neighbors+in+high%2Ddimensional+data%2E+volume+11%2C+pages+2487%E2%80%932531%2E)
24. Sebastian Ruder. 2017. A survey of cross-lingual embedding models. CoRR, abs/1706.04902. [find](https://scholar.google.com/scholar?q=Sebastian+Ruder%2E+2017%2E+A+survey+of+cross%2Dlingual+embedding+models%2E+CoRR%2C+abs%2F1706%2E04902%2E)
25. Samuel L Smith, David HP Turban, Steven Hamblin, and Nils Y Hammerla. 2017. [Offline bilingual word vectors, orthogonal transformations and the inverted softmax](https://md.research.elamur.ai/free/en/papers/smith-2017-offline-bilingual-word-vectors-orthogonal/).
26. Anders Søgaard, Sebastian Ruder, and Ivan Vulić. 2018. [On the limitations of unsupervised bilingual dictionary induction](https://md.research.elamur.ai/free/en/papers/sogaard-2018-limitations-unsupervised-bilingual-dictionary-induction/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 778–788.

*Updated 24 October 2026*