---
title: "Unicoder: A Universal Language Encoder by — Huang 2019"
description: "Pre-trained encoders such as BERT transfer poorly when a task's training data is in one language and its test inputs are in another."
authors: ["Haoyang Huang", "Yaobo Liang", "Nan Duan", "Ming Gong", "Linjun Shou", "Daxin Jiang", "Ming Zhou"]
year: 2019
venue: "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)"
source: "https://aclanthology.org/D19-1252/"
doi: "10.18653/v1/d19-1252"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/huang-2019-unicoder-universal-language-encoder-pre/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/huang-2019-unicoder-universal-language-encoder-pre/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/) / Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks

# Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks

**Authors:** [Haoyang Huang](https://md.research.elamur.ai/free/en/authors/haoyang-huang/), [Yaobo Liang](https://md.research.elamur.ai/free/en/authors/yaobo-liang/), [Nan Duan](https://md.research.elamur.ai/free/en/authors/nan-duan/), [Ming Gong](https://md.research.elamur.ai/free/en/authors/ming-gong/), [Linjun Shou](https://md.research.elamur.ai/free/en/authors/linjun-shou/), [Daxin Jiang](https://md.research.elamur.ai/free/en/authors/daxin-jiang/), [Ming Zhou](https://md.research.elamur.ai/free/en/authors/ming-zhou/) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D19-1252/) · [PDF](https://aclanthology.org/D19-1252.pdf) · [DOI 10.18653/v1/d19-1252](https://doi.org/10.18653/v1/d19-1252) · [OpenAlex](https://openalex.org/W2971207485)

**Citations:** 160 citations (openalex-snapshot · 2026-10-14) · [101](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Pre-trained encoders such as BERT transfer poorly when a task's training data is in one language and its test inputs are in another. Multilingual BERT and XLM partly address this, but XLM relies on a single cross-lingual objective, translation language modeling. Unicoder keeps XLM's architecture and shared BPE vocabulary but pre-trains on five tasks. Alongside masked and translation language modeling, it adds three new cross-lingual objectives, with training data taken from existing machine translation corpora:

- **Cross-lingual word recovery:** an attention-based word alignment objective.
- **Cross-lingual paraphrase classification:** detecting whether two sentences in different languages share a meaning.
- **Cross-lingual masked language model:** masked language modeling over documents that alternate between languages.

The authors also propose Multi-language Fine-tuning, which trains jointly on source-language data and machine-translated data for all target languages. Against an XLM baseline, Unicoder gains 1.8 accuracy points on average across the 15 XNLI languages, a new state of the art. It gains 5.5 points on French and German in XQA, a cross-lingual question answering dataset the authors built. The work shows that richer cross-lingual supervision and multilingual fine-tuning improve zero-resource transfer, even for English.

## Key points
- **Hard negatives for paraphrase classification:** a lightweight DAN model trained on random negatives selects similar but incorrect translations. Positive and negative examples are balanced 1:1.
- **Cross-lingual documents:** these are built from document-aligned MT corpora such as MultiUN and OpenSubtitles. Every even-indexed sentence is replaced by its translation, and the result is truncated to 256 tokens.
- **More fine-tuning languages generally help:** fine-tuning on 15 languages always beats single-language TRANSLATE-TRAIN. Fine-tuning on English alone gives 85.1 on English XNLI with a 75.3 average across languages. Pairing English with French raises these to 85.4 and 76.8.
- **Pairwise effects vary:** adding Vietnamese or Urdu to English lowers average accuracy. Vietnamese and Thai noticeably hurt English, while French and Spanish help it.
- **Transfer patterns across languages:** Russian training data generalizes best, outperforming even English, and Urdu generalizes worst. Transfer among English, Spanish and French is easiest.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

101 papers of our library cite it (of the 140884 whose bibliographies we read); 10 of them have a page here, 91 not yet.

- [On the Cross-lingual Transferability of Monolingual Representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/)
- [ALLaM: Large Language Models for Arabic and English](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/)
- [InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training](https://md.research.elamur.ai/free/en/papers/chi-2020-infoxlm-information-theoretic-framework-cross/)
- [Emerging Cross-lingual Structure in Pretrained Language Models](https://md.research.elamur.ai/free/en/papers/conneau-2020-emerging-cross-lingual-structure-pretrained/)
- [Injecting Numerical Reasoning Skills into Language Models](https://md.research.elamur.ai/free/en/papers/geva-2020-injecting-numerical-reasoning-skills-into/)
- [Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information](https://md.research.elamur.ai/free/en/papers/lin-2020-pre-training-multilingual-neural-machine/)
- [Pre-trained Models for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/)
- [mGPT: Few-Shot Learners Go Multilingual](https://md.research.elamur.ai/free/en/papers/shliazhko-2022-mgpt-few-shot-learners-go/)
- [On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment](https://md.research.elamur.ai/free/en/papers/wang-2020-negative-interference-multilingual-models-findings/)
- [Are All Languages Created Equal in Multilingual BERT?](https://md.research.elamur.ai/free/en/papers/wu-2020-are-all-languages-created-equal/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{huang2019unicoder,
  title = {{Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks}},
  author = {Huang, Haoyang and Liang, Yaobo and Duan, Nan and Gong, Ming and Shou, Linjun and Jiang, Daxin and Zhou, Ming},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1252},
  url = {https://doi.org/10.18653/v1/d19-1252}
}
```

- [← previous Un corpus arboré pour le français : le French Treebank A parsed corpus for French: the French treebank](https://md.research.elamur.ai/free/en/papers/abeille-2019-un-corpus-arbore-pour-le/)
- [next → Unified Language Model Pre-training for Natural Language Understanding and Generation](https://md.research.elamur.ai/free/en/papers/dong-2019-unified-language-model-pre-training/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/)

## References

28 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Proceedings of the Seventh conference on International Language Resources and Evaluation, pages 2868–2872. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Proceedings+of+the+Seventh+conference+on+International+Language+Resources+and+Evaluation%2C+pages+2868%E2%80%932872%2E+European+Language+Resources+Association+%28ELRA%29%2E)
2. Mandy Guo, Qinlan Shen, Yinfei Yang, Heming Ge, Daniel M Cer, Gustavo Hernandez Abrego, Keith Stevens, Noah Constant, Yunhsuan Sung, Brian Strope, et al. 2018. Effective parallel corpus mining using bilingual sentence embeddings. arXiv: Computation and Language, pages 165–176. [find](https://scholar.google.com/scholar?q=Mandy+Guo%2C+Qinlan+Shen%2C+Yinfei+Yang%2C+Heming+Ge%2C+Daniel+M+Cer%2C+Gustavo+Hernandez+Abrego%2C+Keith+Stevens%2C+Noah+Constant%2C+Yunhsuan+Sung%2C+Brian+Strope%2C+et+al%2E+2018%2E+Effective+parallel+corpus+mining+using+b) [our card](https://md.research.elamur.ai/free/en/papers/guo-2018-effective-parallel-corpus-mining-bilingual/)
3. Dan Hendrycks and Kevin Gimpel. 2017. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. arXiv: Learning. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks+and+Kevin+Gimpel%2E+2017%2E+Bridging+nonlinearities+and+stochastic+regularizers+with+gaussian+error+linear+units%2E+arXiv%3A+Learning%2E)
4. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered composition rivals syntactic methods for text classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural L… [find](https://scholar.google.com/scholar?q=Mohit+Iyyer%2C+Varun+Manjunatha%2C+Jordan+Boyd%2DGraber%2C+and+Hal+Daume%CC%81+III%2E+2015%2E+Deep+unordered+composition+rivals+syntactic+methods+for+text+classification%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+t) [our card](https://md.research.elamur.ai/free/en/papers/iyyer-2015-deep-unordered-composition-rivals-syntactic/)
5. Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. international conference on learning representations. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimization%2E+international+conference+on+learning+representations%2E)
6. Waleed Ammar, George Mulcaire, Yulia Tsvetkov, Guillaume Lample, Chris Dyer, and Noah A Smith. 2016. Massively multilingual word embeddings. arXiv preprint arXiv:1602.01925. [find](https://scholar.google.com/scholar?q=Waleed+Ammar%2C+George+Mulcaire%2C+Yulia+Tsvetkov%2C+Guillaume+Lample%2C+Chris+Dyer%2C+and+Noah+A+Smith%2E+2016%2E+Massively+multilingual+word+embeddings%2E+arXiv+preprint+arXiv%3A1602%2E01925%2E) [our card](https://md.research.elamur.ai/free/en/papers/ammar-2016-massively-multilingual-word-embeddings/)
7. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callisonburch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Corbett Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. The Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callisonburch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Corbett+Moran%2C+Richard+Zens%2C+et+al%2E+2007%2E+Moses%3A+Open+source+too)
8. Mikel Artetxe and Holger Schwenk. 2018. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. arXiv preprint arXiv:1812.10464. [find](https://scholar.google.com/scholar?q=Mikel+Artetxe+and+Holger+Schwenk%2E+2018%2E+Massively+multilingual+sentence+embeddings+for+zero%2Dshot+cross%2Dlingual+transfer+and+beyond%2E+arXiv+preprint+arXiv%3A1812%2E10464%2E) [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2018-massively-multilingual-sentence-embeddings-zero/)
9. Anoop Kunchukuttan, Pratik Mehta, and Pushpak Bhattacharyya. 2017. The iit bombay english-hindi parallel corpus. language resources and evaluation. [find](https://scholar.google.com/scholar?q=Anoop+Kunchukuttan%2C+Pratik+Mehta%2C+and+Pushpak+Bhattacharyya%2E+2017%2E+The+iit+bombay+english%2Dhindi+parallel+corpus%2E+language+resources+and+evaluation%2E) [our card](https://md.research.elamur.ai/free/en/papers/kunchukuttan-2018-iit-bombay-english-hindi-parallel/)
10. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+arXiv+preprint+arXiv%3A1409%2E0473%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
11. Pichuan Chang, Michel Galley, and Christopher D Manning. 2008. Optimizing chinese word segmentation for machine translation performance. pages 224–232. [find](https://scholar.google.com/scholar?q=Pichuan+Chang%2C+Michel+Galley%2C+and+Christopher+D+Manning%2E+2008%2E+Optimizing+chinese+word+segmentation+for+machine+translation+performance%2E+pages+224%E2%80%93232%2E) [our card](https://md.research.elamur.ai/free/en/papers/chang-2008-optimizing-chinese-word-segmentation-machine/)
12. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475–2485. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Ruty+Rinott%2C+Guillaume+Lample%2C+Adina+Williams%2C+Samuel+Bowman%2C+Holger+Schwenk%2C+and+Veselin+Stoyanov%2E+2018%2E+Xnli%3A+Evaluating+cross%2Dlingual+sentence+representations%2E+In+Proceedings+of+the)
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+BERT%3A+pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+CoRR%2C+abs%2F1810%2E04805%2E)
14. Andreas Eisele and Yu Chen. 2010. Multiun: A multilingual corpus from united nation documents. In Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. CoRR, abs/1901.07291. [find](https://scholar.google.com/scholar?q=Andreas+Eisele+and+Yu+Chen%2E+2010%2E+Multiun%3A+A+multilingual+corpus+from+united+nation+documents%2E+In+Guillaume+Lample+and+Alexis+Conneau%2E+2019%2E+Cross%2Dlingual+language+model+pretraining%2E+CoRR%2C+abs%2F1901%2E07)
15. Guillaume Lample, Alexis Conneau, Ludovic Denoyer, Hervé Jégou, et al. 2018. Word translation without parallel data. [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Alexis+Conneau%2C+Ludovic+Denoyer%2C+Herve%CC%81+Je%CC%81gou%2C+et+al%2E+2018%2E+Word+translation+without+parallel+data%2E)
16. Liyuan Liu, Jingbo Shang, Frank F Xu, Xiang Ren, Huan Gui, Jian Peng, and Jiawei Han. 2018. Empower sequence labeling with task-aware neural language model. national conference on artificial intelligence, pages 5253–5260. [find](https://scholar.google.com/scholar?q=Liyuan+Liu%2C+Jingbo+Shang%2C+Frank+F+Xu%2C+Xiang+Ren%2C+Huan+Gui%2C+Jian+Peng%2C+and+Jiawei+Han%2E+2018%2E+Empower+sequence+labeling+with+task%2Daware+neural+language+model%2E+national+conference+on+artificial+intellige) [our card](https://md.research.elamur.ai/free/en/papers/liu-2017-empower-sequence-labeling-task-aware/)
17. Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019. Improving multi-task deep neural networks via knowledge distillation for natural language understanding. arXiv: Computation and Language. [find](https://scholar.google.com/scholar?q=Xiaodong+Liu%2C+Pengcheng+He%2C+Weizhu+Chen%2C+and+Jianfeng+Gao%2E+2019%2E+Improving+multi%2Dtask+deep+neural+networks+via+knowledge+distillation+for+natural+language+understanding%2E+arXiv%3A+Computation+and+Languag)
18. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In Advances in Neural Information Processing Systems, pages 6294–6305. [find](https://scholar.google.com/scholar?q=Bryan+McCann%2C+James+Bradbury%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2017%2E+Learned+in+translation%3A+Contextualized+word+vectors%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+6294%E2%80%936305%2E) [our card](https://md.research.elamur.ai/free/en/papers/mccann-2017-learned-translation-contextualized-word-vectors/)
19. Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013. Exploiting similarities among languages for machine translation. arXiv: Computation and Language. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Quoc+V+Le%2C+and+Ilya+Sutskever%2E+2013%2E+Exploiting+similarities+among+languages+for+machine+translation%2E+arXiv%3A+Computation+and+Language%2E)
20. Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10):1345–1359. [find](https://scholar.google.com/scholar?q=Sinno+Jialin+Pan+and+Qiang+Yang%2E+2010%2E+A+survey+on+transfer+learning%2E+IEEE+Transactions+on+Knowledge+and+Data+Engineering%2C+22%2810%29%3A1345%E2%80%931359%2E)
21. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… [find](https://scholar.google.com/scholar?q=Matthew+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+In+Proceedings+of+the+2018+Conference+of+t)
22. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL [https://s3-us-west-2.amazonaws](https://s3-us-west-2.amazonaws). com/openai-assets/research-covers/languageunsupervised/language understanding paper. pdf. [↗](https://s3-us-west-2.amazonaws)
23. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1715–1725. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Ling)
24. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)
25. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. pages 353–355. [find](https://scholar.google.com/scholar?q=Alex+Wang%2C+Amanpreet+Singh%2C+Julian+Michael%2C+Felix+Hill%2C+Omer+Levy%2C+and+Samuel+R+Bowman%2E+2018%2E+Glue%3A+A+multi%2Dtask+benchmark+and+analysis+platform+for+natural+language+understanding%2E+pages+353%E2%80%93355%2E) [our card](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/)
26. Adina Williams, Nikita Nangia, and Samuel R Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. north american chapter of the association for computational linguistics, 1:1112–1122. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+R+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+north+american+chapter+of+the+association+for+computational+li)
27. Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas: The surprising cross-lingual effectiveness of bert. arXiv preprint arXiv:1904.09077. [find](https://scholar.google.com/scholar?q=Shijie+Wu+and+Mark+Dredze%2E+2019%2E+Beto%2C+bentz%2C+becas%3A+The+surprising+cross%2Dlingual+effectiveness+of+bert%2E+arXiv+preprint+arXiv%3A1904%2E09077%2E) [our card](https://md.research.elamur.ai/free/en/papers/wu-2019-beto-bentz-becas-surprising-cross/)
28. Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. 2015. Normalized word embedding and orthogonal transform for bilingual word translation. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1006–1011. [find](https://scholar.google.com/scholar?q=Chao+Xing%2C+Dong+Wang%2C+Chao+Liu%2C+and+Yiye+Lin%2E+2015%2E+Normalized+word+embedding+and+orthogonal+transform+for+bilingual+word+translation%2E+In+Proceedings+of+the+2015+Conference+of+the+North+American+Chapt) [our card](https://md.research.elamur.ai/free/en/papers/xing-2015-normalized-word-embedding-orthogonal-transform/)