---
title: "Gromov-Wasserstein Alignment of Word Embedding Spaces"
description: "The paper addresses unsupervised bilingual lexical induction: matching words across two languages… 32 references, cited by 62 papers here."
authors: ["David Alvarez-Melis", "Tommi Jaakkola"]
year: 2018
venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D18-1214/"
doi: "10.18653/v1/d18-1214"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/melis-2018-gromov-wasserstein-alignment-word-embedding/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/melis-2018-gromov-wasserstein-alignment-word-embedding/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / Gromov-Wasserstein Alignment of Word Embedding Spaces

# Gromov-Wasserstein Alignment of Word Embedding Spaces

**Authors:** David Alvarez-Melis and [Tommi Jaakkola](https://md.research.elamur.ai/free/en/authors/tommi-jaakkola/) · **Year:** 2018 · **Venue:** [Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2018](https://md.research.elamur.ai/free/en/venues/emnlp/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D18-1214/) · [PDF](https://aclanthology.org/D18-1214.pdf) · [DOI 10.18653/v1/d18-1214](https://doi.org/10.18653/v1/d18-1214) · [OpenAlex](https://openalex.org/W2892266277)

**Citations:** 275 citations (OpenAlex, 2026-10-14) · [62](https://md.research.elamur.ai/free/en/papers/melis-2018-gromov-wasserstein-alignment-word-embedding/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper addresses unsupervised bilingual lexical induction: matching words across two languages using only monolingual embeddings, with no parallel data. Earlier fully unsupervised systems chain several stages, such as adversarial initialization, Procrustes fitting and heuristic refinement, and they can be hard to tune. The authors observe that embeddings mostly encode relations between words, not absolute positions. They therefore frame alignment as a Gromov-Wasserstein optimal transport problem, which compares within-language distance structures directly. The problem is solved with an entropy-regularized, Sinkhorn-style procedure. To handle large vocabularies, the coupling learned on a subset of words is used to fit an orthogonal Procrustes map, which then covers the remaining words. On standard word translation benchmarks, including the Dinu et al. (2014) dataset, the method comes close to state-of-the-art accuracy. It does so in a single optimization step, with little hyperparameter tuning, using only CPU computation and far less compute than adversarial alternatives. The optimal GW value also serves as a geometric distance between languages.

## Key points
- The core objective is the Gromov-Wasserstein distance (Mémoli, 2011), a quadratic transport problem over pairs of words. It is optimized with entropic regularization following Cuturi (2013) and needs no post-hoc refinement.
- The method scales by learning a single orthogonal Procrustes mapping from the GW coupling and applying it to out-of-sample words.
- Its runtimes are CPU-minutes. The baselines reported by Artetxe et al. (2018) were run on a Titan XP GPU, so the authors note the timings are not strictly comparable.
- The authors computed pairwise language distances on the FastText embeddings of Conneau et al. (2018), using the 2000 most frequent words for DE, EN, ES, FR, IT, RU and ZH.
- In those distances, Romance languages cluster closely and ZH is farthest from all others. RU is unexpectedly near the Romance group, with ES its closest neighbour, while EN sits roughly equidistant from DE and FR.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 159 works · 65,157 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 54 works · 29,441 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: David Alvarez-Melis and Tommi Jaakkola. 2018. Gromov-Wasserstein Alignment of Word Embedding Spaces. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{alvarezmelis2018gromov,
  title = {{Gromov-Wasserstein Alignment of Word Embedding Spaces}},
  author = {David Alvarez-Melis and Tommi Jaakkola},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1214},
  url = {https://doi.org/10.18653/v1/d18-1214}
}
```

- [← previous A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss](https://md.research.elamur.ai/free/en/papers/hsu-2018-unified-model-extractive-abstractive-summarization/)
- [next → Predicting Factuality of Reporting and Bias of News Media Sources](https://md.research.elamur.ai/free/en/papers/baly-2018-predicting-factuality-reporting-bias-news/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/)

## References

32 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2016. Learning principled bilingual mappings of word embeddings while preserving monolingual invariance. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2289–2294. [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2016-learning-principled-bilingual-mappings-word/)
2. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018. A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 789—-798. Association for Com… [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2018-vecmap-unsupervised/)
3. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5:135–146. [our card](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/)
4. Georg Bossong. 1991. Differential object marking in Romance and beyond. New analyses in Romance linguistics, pages 143–170. [find](https://scholar.google.com/scholar?q=Georg+Bossong%2E+1991%2E+Differential+object+marking+in+Romance+and+beyond%2E+New+analyses+in+Romance+linguistics%2C+pages+143%E2%80%93170%2E)
5. Alexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018. Word Translation Without Parallel Data. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Guillaume+Lample%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Ludovic+Denoyer%2C+and+Herve%CC%81+Je%CC%81gou%2E+2018%2E+Word+Translation+Without+Parallel+Data%2E+In+International+Conference+on+Learning+Representations%2E)
6. Marco Cuturi. 2013. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in Neural Information Processing Systems, pages 2292—-2300. [find](https://scholar.google.com/scholar?q=Marco+Cuturi%2E+2013%2E+Sinkhorn+distances%3A+Lightspeed+computation+of+optimal+transport%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+2292%E2%80%94%2D2300%2E)
7. Georgiana Dinu, Angeliki Lazaridou, and Marco Baroni. 2014. Improving zero-shot learning by mitigating the hubness problem. arXiv preprint arXiv:1412.6568. — [arXiv:1412.6568](https://arxiv.org/abs/1412.6568) [find](https://scholar.google.com/scholar?q=Georgiana+Dinu%2C+Angeliki+Lazaridou%2C+and+Marco+Baroni%2E+2014%2E+Improving+zero%2Dshot+learning+by+mitigating+the+hubness+problem%2E+arXiv+preprint+arXiv%3A1412%2E6568%2E+%E2%80%94+%5BarXiv%3A1412%2E6568%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F14)
8. Manaal Faruqui and Chris Dyer. 2014. Improving vector space word representations using multilingual correlation. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 462–471. [our card](https://md.research.elamur.ai/free/en/papers/faruqui-2014-improving-vector-space-word-representations/)
9. Pascale Fung. 1995. Compiling bilingual lexicon entries from a non-parallel English-Chinese corpus. In Third Workshop on Very Large Corpora. [find](https://scholar.google.com/scholar?q=Pascale+Fung%2E+1995%2E+Compiling+bilingual+lexicon+entries+from+a+non%2Dparallel+English%2DChinese+corpus%2E+In+Third+Workshop+on+Very+Large+Corpora%2E)
10. Jiang Guo, Wanxiang Che, David Yarowsky, Haifeng Wang, and Ting Liu. 2015. Cross-lingual dependency parsing based on distributed representations. [our card](https://md.research.elamur.ai/free/en/papers/guo-2015-cross-lingual-dependency-parsing-based/)
11. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 1234–1244. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+7th+International+Joint+Conference+on+Natural+Language+Processing+%28Volume+1%3A+Long+Papers%29%2C+volume+1%2C+)
12. Tatsunori B Hashimoto, David Alvarez-Melis, and Tommi S Jaakkola. 2016. Word Embeddings as Metric Recovery in Semantic Spaces. Transactions of the Association for Computational Linguistics, 4:273–286. [find](https://scholar.google.com/scholar?q=Tatsunori+B+Hashimoto%2C+David+Alvarez%2DMelis%2C+and+Tommi+S+Jaakkola%2E+2016%2E+Word+Embeddings+as+Metric+Recovery+in+Semantic+Spaces%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+4%3A273%E2%80%93286%2E)
13. Martin Haspelmath. 2001. The European linguistic area: Standard Average European. In Language typology and language universals: An international handbook, volume 2, pages 1492–1510. de Gruyter. [find](https://scholar.google.com/scholar?q=Martin+Haspelmath%2E+2001%2E+The+European+linguistic+area%3A+Standard+Average+European%2E+In+Language+typology+and+language+universals%3A+An+international+handbook%2C+volume+2%2C+pages+1492%E2%80%931510%2E+de+Gruyter%2E)
14. Guillaume Lample, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2018. Unsupervised Machine Translation Using Monolingual Corpora Only. International Conference on Learning Representations. [our card](https://md.research.elamur.ai/free/en/papers/lample-2017-unsupervised-machine-translation-using-monolingual/)
15. Ang Lu, Weiran Wang, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2015. Deep multilingual correlation for improved word embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 250–256. [find](https://scholar.google.com/scholar?q=Ang+Lu%2C+Weiran+Wang%2C+Mohit+Bansal%2C+Kevin+Gimpel%2C+and+Karen+Livescu%2E+2015%2E+Deep+multilingual+correlation+for+improved+word+embeddings%2E+In+Proceedings+of+the+2015+Conference+of+the+North+American+Chapte)
16. Chantal Melis, Marcela Flores, and A Holvoet. 2013. On the historical expansion of non-canonically marked ‘subjects’ in Spanish. The diachronic Typology of Non-Canonical Subjects, Amsterdam/Philadelphia, Benjamins, pages 163–184. [find](https://scholar.google.com/scholar?q=Chantal+Melis%2C+Marcela+Flores%2C+and+A+Holvoet%2E+2013%2E+On+the+historical+expansion+of+non%2Dcanonically+marked+%E2%80%98subjects%E2%80%99+in+Spanish%2E+The+diachronic+Typology+of+Non%2DCanonical+Subjects%2C+Amsterdam%2FPhiladelph)
17. Facundo Mémoli. 2011. Gromov–Wasserstein distances and the metric approach to object matching. Foundations of computational mathematics, 11(4):417–487. [find](https://scholar.google.com/scholar?q=Facundo+Me%CC%81moli%2E+2011%2E+Gromov%E2%80%93Wasserstein+distances+and+the+metric+approach+to+object+matching%2E+Foundations+of+computational+mathematics%2C+11%284%29%3A417%E2%80%93487%2E)
18. Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013. Exploiting Similarities among Languages for Machine Translation. arXiv preprint arXiv:1309.4168v1, pages 1–10. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Quoc+V+Le%2C+and+Ilya+Sutskever%2E+2013%2E+Exploiting+Similarities+among+Languages+for+Machine+Translation%2E+arXiv+preprint+arXiv%3A1309%2E4168v1%2C+pages+1%E2%80%9310%2E)
19. John Moore and David M. Perlmutter. 2000. What does it take to be a dative subject? Natural Language and Linguistic Theory, 18(2):373–416. [find](https://scholar.google.com/scholar?q=John+Moore+and+David+M%2E+Perlmutter%2E+2000%2E+What+does+it+take+to+be+a+dative+subject%3F+Natural+Language+and+Linguistic+Theory%2C+18%282%29%3A373%E2%80%93416%2E)
20. Gabriel Peyré and Marco Cuturi. 2018. Computational Optimal Transport. Technical report. [find](https://scholar.google.com/scholar?q=Gabriel+Peyre%CC%81+and+Marco+Cuturi%2E+2018%2E+Computational+Optimal+Transport%2E+Technical+report%2E)
21. Gabriel Peyré, Marco Cuturi, and Justin Solomon. 2016. Gromov-Wasserstein averaging of kernel and distance matrices. In International Conference on Machine Learning, pages 2664–2672. [find](https://scholar.google.com/scholar?q=Gabriel+Peyre%CC%81%2C+Marco+Cuturi%2C+and+Justin+Solomon%2E+2016%2E+Gromov%2DWasserstein+averaging+of+kernel+and+distance+matrices%2E+In+International+Conference+on+Machine+Learning%2C+pages+2664%E2%80%932672%2E)
22. Anand Rangarajan, Haili Chui, and Fred L Bookstein. 1997. The Softassign Procrustes Matching Algorithm. Lecture Notes in Computer Science, 1230:29–42. [find](https://scholar.google.com/scholar?q=Anand+Rangarajan%2C+Haili+Chui%2C+and+Fred+L+Bookstein%2E+1997%2E+The+Softassign+Procrustes+Matching+Algorithm%2E+Lecture+Notes+in+Computer+Science%2C+1230%3A29%E2%80%9342%2E)
23. Reinhard Rapp. 1995. Identifying word translations in non-parallel texts. In Proceedings of the 33rd annual meeting on Association for Computational Linguistics, pages 320–322. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/rapp-1995-identifying-word-translations-non-parallel/)
24. Reinhard Rapp. 1999. Automatic identification of word translations from unrelated English and German corpora. In Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, pages 519– 526. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/rapp-1999-automatic-identification-word-translations-unrelated/)
25. Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2017. A survey of cross-lingual embedding models. arXiv preprint arXiv:1706.04902. [find](https://scholar.google.com/scholar?q=Sebastian+Ruder%2C+Ivan+Vulic%CC%81%2C+and+Anders+S%C3%B8gaard%2E+2017%2E+A+survey+of+cross%2Dlingual+embedding+models%2E+arXiv+preprint+arXiv%3A1706%2E04902%2E)
26. Peter H. Schönemann. 1966. A generalized solution of the orthogonal procrustes problem. Psychometrika, 31(1):1–10. [find](https://scholar.google.com/scholar?q=Peter+H%2E+Scho%CC%88nemann%2E+1966%2E+A+generalized+solution+of+the+orthogonal+procrustes+problem%2E+Psychometrika%2C+31%281%29%3A1%E2%80%9310%2E)
27. Samuel L Smith, David H P Turban, Steven Hamblin, and Nils Y Hammerla. 2017. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. International Conference on Learning Representations. [our card](https://md.research.elamur.ai/free/en/papers/smith-2017-offline-bilingual-word-vectors-orthogonal/)
28. Shyam Upadhyay, Manaal Faruqui, Chris Dyer, and Dan Roth. 2016. Cross-lingual models of word embeddings: An empirical comparison. arXiv preprint arXiv:1604.00425. [our card](https://md.research.elamur.ai/free/en/papers/upadhyay-2016-cross-lingual-models-word-embeddings/)
29. Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017a. Adversarial training for unsupervised bilingual lexicon induction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1959–1970. [our card](https://md.research.elamur.ai/free/en/papers/zhang-2017-adversarial-training-unsupervised-bilingual-lexicon/)
30. Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017b. Earth Mover’s Distance Minimization for Unsupervised Bilingual Lexicon Induction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1934–1945. [find](https://scholar.google.com/scholar?q=Meng+Zhang%2C+Yang+Liu%2C+Huanbo+Luan%2C+and+Maosong+Sun%2E+2017b%2E+Earth+Mover%E2%80%99s+Distance+Minimization+for+Unsupervised+Bilingual+Lexicon+Induction%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+)
31. Yuan Zhang, David Gaddy, Regina Barzilay, and Tommi Jaakkola. 2016. Ten Pairs to Tag – Multilingual POS Tagging via Coarse Mapping between Embeddings. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pa… [find](https://scholar.google.com/scholar?q=Yuan+Zhang%2C+David+Gaddy%2C+Regina+Barzilay%2C+and+Tommi+Jaakkola%2E+2016%2E+Ten+Pairs+to+Tag+%E2%80%93+Multilingual+POS+Tagging+via+Coarse+Mapping+between+Embeddings%2E+In+Proceedings+of+the+2016+Conference+of+the+Nort)
32. Steven T Piantadosi. 2014. Zipf’s word frequency law in natural language: A critical review and future directions. Psychonomic Bulletin & Review, 21(5):1112–1130. [find](https://scholar.google.com/scholar?q=Steven+T+Piantadosi%2E+2014%2E+Zipf%E2%80%99s+word+frequency+law+in+natural+language%3A+A+critical+review+and+future+directions%2E+Psychonomic+Bulletin+%26+Review%2C+21%285%29%3A1112%E2%80%931130%2E)

*Updated 24 October 2026*