--- title: "Learning principled bilingual mappings of — Artetxe 2016" description: "The paper addresses cross-lingual word embedding mapping: learning a linear transformation that maps independently trained source-language vectors into a…" authors: ["Mikel Artetxe", "Gorka Labaka", "Eneko Agirre"] year: 2016 venue: "Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/D16-1250/" doi: "10.18653/v1/d16-1250" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/artetxe-2016-learning-principled-bilingual-mappings-word/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/artetxe-2016-learning-principled-bilingual-mappings-word/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/) / Learning principled bilingual mappings of word embeddings while preserving monolingual invariance # Learning principled bilingual mappings of word embeddings while preserving monolingual invariance **Authors:** [Mikel Artetxe](https://md.research.elamur.ai/free/en/authors/mikel-artetxe/), [Gorka Labaka](https://md.research.elamur.ai/free/en/authors/gorka-labaka/), [Eneko Agirre](https://md.research.elamur.ai/free/en/authors/eneko-agirre/) · **Year:** 2016 · **Venue:** Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/D16-1250/) · [PDF](https://aclanthology.org/D16-1250.pdf) · [DOI 10.18653/v1/d16-1250](https://doi.org/10.18653/v1/d16-1250) · [OpenAlex](https://openalex.org/W2561995736) **Citations:** 369 citations (openalex-snapshot · 2026-10-14) · [164](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary The paper addresses cross-lingual word embedding mapping: learning a linear transformation that maps independently trained source-language vectors into a target-language space, using a bilingual dictionary as supervision. Artetxe, Labaka and Agirre start from the least-squares objective of Mikolov et al. (2013b). They then add three constraints: an orthogonal mapping, length normalization, and dimension-wise mean centering. Orthogonality keeps dot products unchanged, so the source space stays intact for monolingual tasks. The resulting framework yields an exact, SVD-based closed-form solution instead of gradient descent. It also reframes earlier methods. Xing et al. (2015) amounts to an orthogonal mapping on unit-length vectors. Faruqui and Dyer's CCA approach is close to the full model but also forces dimensions to be uncorrelated with equal variance. The authors evaluate English–Italian translation induction and the English word analogy task. The full configuration achieves the best translation accuracy while leaving analogy performance almost unchanged. This shows that orthogonality, more than normalization, drives the improvements. ## Key points - **Data:** The Dinu et al. (2015) English–Italian dataset provides 300-dimensional word2vec CBOW embeddings. English vectors come from ukWaC + Wikipedia + BNC (2.8B words) and Italian vectors from itWaC (1.6B words). It includes a Europarl-derived dictionary with 5,000 training pairs and 1,500 test pairs. - **Translation induction (EN-IT accuracy):** - Mikolov et al. (2013b): 34.93% - Xing et al. (2015): 36.87% - Faruqui and Dyer (2014), CCA with 180 dimensions: 37.80% - Proposed method (orthogonal + length normalization + mean centering): 39.27% - **Monolingual analogy (Mikolov et al. 2013a questions, 30,000-word vocabulary threshold):** The original embeddings score 76.66%. The proposed method loses only 0.07 points, compared with 2.86 for Mikolov et al. and 7.02 for CCA. - **Exact solution:** The optimal orthogonal mapping is W = VUᵀ, obtained from the SVD of ZᵀX. Its cost grows linearly with vocabulary size. - **Code:** The open-source Python/NumPy implementation is released as **vecmap** (github.com/artetxem/vecmap). *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 164 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 155 not yet. - [Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling](https://md.research.elamur.ai/free/en/papers/acl-2019-zalmout-adversarial-multitask-learning-joint-multi/) - [Learning bilingual word embeddings with (almost) no bilingual data](https://md.research.elamur.ai/free/en/papers/artetxe-2017-learning-bilingual-word-embeddings-almost/) - [Unsupervised Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/artetxe-2017-unsupervised-neural-machine-translation/) - [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://md.research.elamur.ai/free/en/papers/artetxe-2018-vecmap-unsupervised/) - [Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion](https://md.research.elamur.ai/free/en/papers/joulin-2018-loss-translation-learning-bilingual-word/) - [Few-shot Prompting or Supervised Tuning? A Comparative Study of LLMs for Linguistically Distant Language Pairs in BDI](https://md.research.elamur.ai/free/en/papers/naorem-2026-few-shot-prompting-or-supervised/) - [Offline bilingual word vectors, orthogonal transformations and the inverted softmax](https://md.research.elamur.ai/free/en/papers/smith-2017-offline-bilingual-word-vectors-orthogonal/) - [Neural Cross-Lingual Named Entity Recognition with Minimal Resources](https://md.research.elamur.ai/free/en/papers/xie-2018-neural-cross-lingual-named-entity/) - [Adversarial Training for Unsupervised Bilingual Lexicon Induction](https://md.research.elamur.ai/free/en/papers/zhang-2017-adversarial-training-unsupervised-bilingual-lexicon/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{artetxe2016learning, title = {{Learning principled bilingual mappings of word embeddings while preserving monolingual invariance}}, author = {Artetxe, Mikel and Labaka, Gorka and Agirre, Eneko}, booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing}, year = {2016}, doi = {10.18653/v1/d16-1250}, url = {https://doi.org/10.18653/v1/d16-1250} } ``` - [← previous Learning Natural Language Inference with LSTM](https://md.research.elamur.ai/free/en/papers/wang-2016-learning-natural-language-inference-lstm/) - [next → Learning to Compose Neural Networks for Question Answering](https://md.research.elamur.ai/free/en/papers/andreas-2016-learning-compose-neural-networks-question/) - [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/) ## References 8 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Sarath Chandar A P, Stanislas Lauly, Hugo Larochelle, Mitesh Khapra, Balaraman Ravindran, Vikas C Raykar, and Amrita Saha. 2014. An autoencoder approach to learning bilingual word representations. In Advances in Neural Information Processing Systems Acknowledgments 27, pages 1853–1861. Georgiana Din… [find](https://scholar.google.com/scholar?q=Sarath+Chandar+A+P%2C+Stanislas+Lauly%2C+Hugo+Larochelle%2C+Mitesh+Khapra%2C+Balaraman+Ravindran%2C+Vikas+C+Raykar%2C+and+Amrita+Saha%2E+2014%2E+An+autoencoder+approach+to+learning+bilingual+word+representations%2E+In+) 2. Min-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Bilingual word representations with monolingual quality in mind. In NAACL Workshop on Vector Space Modeling for NLP, pages 151–159. [find](https://scholar.google.com/scholar?q=Min%2DThang+Luong%2C+Hieu+Pham%2C+and+Christopher+D%2E+Manning%2E+2015%2E+Bilingual+word+representations+with+monolingual+quality+in+mind%2E+In+NAACL+Workshop+on+Vector+Space+Modeling+for+NLP%2C+pages+151%E2%80%93159%2E) [our card](https://md.research.elamur.ai/free/en/papers/luong-2015-bilingual-word-representations-monolingual-quality/) 3. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeffrey+Dean%2E+2013a%2E+Efficient+estimation+of+word+representations+in+vector+space%2E+arXiv+preprint+arXiv%3A1301%2E3781%2E) 4. Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013b. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Quoc+V+Le%2C+and+Ilya+Sutskever%2E+2013b%2E+Exploiting+similarities+among+languages+for+machine+translation%2E+arXiv+preprint+arXiv%3A1309%2E4168%2E) 5. Michael Wick, Pallika Kanani, and Adam Pocock. 2016. Minimally-constrained multilingual embeddings via artificial code-switching. In Thirtieth AAAI conference on Artificial Intelligence (AAAI). [find](https://scholar.google.com/scholar?q=Michael+Wick%2C+Pallika+Kanani%2C+and+Adam+Pocock%2E+2016%2E+Minimally%2Dconstrained+multilingual+embeddings+via+artificial+code%2Dswitching%2E+In+Thirtieth+AAAI+conference+on+Artificial+Intelligence+%28AAAI%29%2E) 6. Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. 2015. Normalized word embedding and orthogonal transform for bilingual word translation. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1006–1011. [find](https://scholar.google.com/scholar?q=Chao+Xing%2C+Dong+Wang%2C+Chao+Liu%2C+and+Yiye+Lin%2E+2015%2E+Normalized+word+embedding+and+orthogonal+transform+for+bilingual+word+translation%2E+In+Proceedings+of+the+2015+Conference+of+the+North+American+Chapt) [our card](https://md.research.elamur.ai/free/en/papers/xing-2015-normalized-word-embedding-orthogonal-transform/) 7. Kai Zhao, Hany Hassan, and Michael Auli. 2015. Learning translation models from monolingual continuous representations. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1527–1536. [find](https://scholar.google.com/scholar?q=Kai+Zhao%2C+Hany+Hassan%2C+and+Michael+Auli%2E+2015%2E+Learning+translation+models+from+monolingual+continuous+representations%2E+In+Proceedings+of+the+2015+Conference+of+the+North+American+Chapter+of+the+Assoc) 8. Will Y. Zou, Richard Socher, Daniel Cer, and Christopher D. Manning. 2013. Bilingual word embeddings for phrase-based machine translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1393–1398. [find](https://scholar.google.com/scholar?q=Will+Y%2E+Zou%2C+Richard+Socher%2C+Daniel+Cer%2C+and+Christopher+D%2E+Manning%2E+2013%2E+Bilingual+word+embeddings+for+phrase%2Dbased+machine+translation%2E+In+Proceedings+of+the+2013+Conference+on+Empirical+Methods+in) [our card](https://md.research.elamur.ai/free/en/papers/zou-2013-bilingual-word-embeddings-phrase-based/)