---
title: "Cross-Lingual Alignment of Contextual Word Embeddings, with Applications — Schuster 2019"
description: "Contextual embeddings such as ELMo produce a different vector for each occurrence of a word. 8.8 LAS. 62 references, cited by 112 papers here."
authors: ["Tal Schuster", "Ori Ram", "Regina Barzilay", "Amir Globerson"]
year: 2019
venue: "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"
source: "https://aclanthology.org/N19-1162/"
doi: "10.18653/v1/n19-1162"
arxiv: "1902.09492"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/schuster-2019-cross-lingual-alignment-contextual-word/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/schuster-2019-cross-lingual-alignment-contextual-word/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/) / Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing

# Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing

**Authors:** [Tal Schuster](https://md.research.elamur.ai/free/en/authors/tal-schuster/), [Ori Ram](https://md.research.elamur.ai/free/en/authors/ori-ram/), [Regina Barzilay](https://md.research.elamur.ai/free/en/authors/regina-barzilay/) and [Amir Globerson](https://md.research.elamur.ai/free/en/authors/amir-globerson/) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N19-1162/) · [PDF](https://aclanthology.org/N19-1162.pdf) · [DOI 10.18653/v1/n19-1162](https://doi.org/10.18653/v1/n19-1162) · [arXiv 1902.09492](https://arxiv.org/abs/1902.09492) · [OpenAlex](https://openalex.org/W2915128308)

**Citations:** 182 citations (OpenAlex, 2026-10-14) · [112](https://md.research.elamur.ai/free/en/papers/schuster-2019-cross-lingual-alignment-contextual-word/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
Contextual embeddings such as ELMo produce a different vector for each occurrence of a word. This makes cross-lingual alignment harder than with static embeddings, because a translation pair no longer corresponds to one vector per language. The authors work around this by computing an "anchor" for each token, which is its contextual vectors averaged over unlabeled text. They show that a word's occurrences cluster tightly around its anchor, even for homonyms.

An orthogonal linear map is learned between the anchor spaces of two languages, either from a bilingual dictionary or without supervision. That same map is then applied to the full contextual embeddings. For languages with little raw text, they also add an alignment term to the language-model loss that pulls dictionary-translation anchors together.

On zero-shot dependency parsing, a parser trained on other languages with aligned ELMo embeddings beats prior state-of-the-art by a wide margin. It also helps in few-shot settings. This matters for transferring parsers to low-resource languages that lack treebanks.

## Key points
- **Anchor geometry (ELMo):**
  - The average cosine distance between anchors of different words is about 0.85.
  - Contextual vectors sit only about 0.18 from their own anchor.
  - For homonyms this distance is about 0.21, which is still close.
- **Zero-shot, multi-source parsing:**
  - Tested on six languages drawn from version 2.0 of the Google universal dependency treebank, with training on English plus the other five.
  - The method gains 6.8 LAS on average over Ammar et al. (2016).
  - Without POS tags it still wins in 5 of 6 languages; Portuguese is the exception, dropping 8.8 LAS.
  - It beats fastText embeddings by about 3 LAS on average.
- **Few-shot parsing on the Kazakh treebank (CoNLL 2018):**
  - The training set has only 38 trees, supplemented with Turkish data and a FastAlign/OpenSubtitles2018 dictionary.
  - The method scores 5.05 LAS above the best shared-task system and more than 10 points above fastText.
- **Limited raw data:**
  - Simulated with only 10k Spanish sentences.
  - The anchored language model improves perplexity and adds about 7 UAS and 9 LAS.
- **Availability:** code and models are released at github.com/TalSchuster/CrossLingualELMo.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [🌳 Dependency parsing](https://md.research.elamur.ai/free/en/topics/dependency-parsing/) · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/)

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 125 works · 60,031 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 45 works · 26,807 citations
- [Tel Aviv University](https://md.research.elamur.ai/free/en/universities/tel-aviv-university/) — University, Tel Aviv, Israel · 54 works · 7,966 citations

## Cite

Cite as: Tal Schuster, Ori Ram, Regina Barzilay and Amir Globerson. 2019. Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{schuster2019cross,
  title = {{Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing}},
  author = {Tal Schuster, Ori Ram, Regina Barzilay and Amir Globerson},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1162},
  eprint = {1902.09492},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n19-1162}
}
```

- [← previous Towards Debiasing Fact Verification Models](https://md.research.elamur.ai/free/en/papers/schuster-2019-debiasing-fact-verification-models/)
- [next → Matching the Blanks: Distributional Similarity for Relation Learning](https://md.research.elamur.ai/free/en/papers/soares-2019-matching-blanks-distributional-similarity-relation/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/)

## References

62 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Oliver Adams, Adam Makarucha, Graham Neubig, Steven Bird, and Trevor Cohn. 2017. Cross-lingual word embeddings for low-resource language modeling. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 937–947. Ass… [find](https://scholar.google.com/scholar?q=Oliver+Adams%2C+Adam+Makarucha%2C+Graham+Neubig%2C+Steven+Bird%2C+and+Trevor+Cohn%2E+2017%2E+Cross%2Dlingual+word+embeddings+for+low%2Dresource+language+modeling%2E+In+Proceedings+of+the+15th+Conference+of+the+European)
2. Hanan Aldarmaki and Mona Diab. 2019. Context-aware crosslingual mapping. arXiv preprint arXiv:1903.03243. — [arXiv:1903.03243](https://arxiv.org/abs/1903.03243) [find](https://scholar.google.com/scholar?q=Hanan+Aldarmaki+and+Mona+Diab%2E+2019%2E+Context%2Daware+crosslingual+mapping%2E+arXiv+preprint+arXiv%3A1903%2E03243%2E+%E2%80%94+%5BarXiv%3A1903%2E03243%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1903%2E03243%29)
3. David Alvarez-Melis and Tommi Jaakkola. 2018. Gromov-Wasserstein alignment of word embedding spaces. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1881–1890. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=David+Alvarez%2DMelis+and+Tommi+Jaakkola%2E+2018%2E+Gromov%2DWasserstein+alignment+of+word+embedding+spaces%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+18)
4. Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah Smith. 2016. Many languages, one parser. Transactions of the Association for Computational Linguistics, 4:431–444. [our card](https://md.research.elamur.ai/free/en/papers/ammar-2016-many-languages-one-parser/)
5. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2017. Learning bilingual word embeddings with (almost) no bilingual data. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 451–462. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2017-learning-bilingual-word-embeddings-almost/)
6. Yonatan Belinkov, Lluı́s Màrquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2017. Evaluating layers of representation in neural machine translation on part-of-speech and semantic tagging tasks. In Proceedings of the Eighth International Joint Conference on Natural Language Process… [our card](https://md.research.elamur.ai/free/en/papers/belinkov-2018-evaluating-layers-representation-neural-machine/)
7. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. [our card](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/)
8. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 644–648. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Language+Technologies%2C+pages+644%E2%80%93648%2E+Association+for+Computational+Linguistics%2E)
9. Wanxiang Che, Yijia Liu, Yuxuan Wang, Bo Zheng, and Ting Liu. 2018. Towards better UD parsing: Deep contextualized word embeddings, ensemble, and treebank concatenation. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 55–64. Associati… [find](https://scholar.google.com/scholar?q=Wanxiang+Che%2C+Yijia+Liu%2C+Yuxuan+Wang%2C+Bo+Zheng%2C+and+Ting+Liu%2E+2018%2E+Towards+better+UD+parsing%3A+Deep+contextualized+word+embeddings%2C+ensemble%2C+and+treebank+concatenation%2E+In+Proceedings+of+the+CoNLL+20)
10. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 1–6. Association… [our card](https://md.research.elamur.ai/free/en/papers/gardner-2018-allennlp-deep-semantic-natural-language/)
11. Kevin Clark, Minh-Thang Luong, Christopher D. Manning, and Quoc Le. 2018. Semi-supervised sequence modeling with cross-view training. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1914– 1925. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/clark-2018-semi-supervised-sequence-modeling-cross/)
12. Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018a. Learning word vectors for 157 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). [our card](https://md.research.elamur.ai/free/en/papers/grave-2018-learning-word-vectors-157-languages/)
13. Alexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018a. Word translation without parallel data. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Guillaume+Lample%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Ludovic+Denoyer%2C+and+Herve%CC%81+Je%CC%81gou%2E+2018a%2E+Word+translation+without+parallel+data%2E+In+International+Conference+on+Learning+Representations%2E)
14. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018b. XNLI: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475–2485. Association… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Ruty+Rinott%2C+Guillaume+Lample%2C+Adina+Williams%2C+Samuel+Bowman%2C+Holger+Schwenk%2C+and+Veselin+Stoyanov%2E+2018b%2E+XNLI%3A+Evaluating+cross%2Dlingual+sentence+representations%2E+In+Proceedings+of+th)
15. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
16. Georgiana Dinu and Marco Baroni. 2014. Improving zero-shot learning by mitigating the hubness problem. CoRR, abs/1412.6568. — [arXiv:1412.6568](https://arxiv.org/abs/1412.6568) [find](https://scholar.google.com/scholar?q=Georgiana+Dinu+and+Marco+Baroni%2E+2014%2E+Improving+zero%2Dshot+learning+by+mitigating+the+hubness+problem%2E+CoRR%2C+abs%2F1412%2E6568%2E+%E2%80%94+%5BarXiv%3A1412%2E6568%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1412%2E6568%29)
17. Timothy Dozat and Christopher D. Manning. 2017. Deep Biaffine Attention for Neural Dependency Parsing. In International Conference on Learning Representations. [our card](https://md.research.elamur.ai/free/en/papers/dozat-2016-deep-biaffine-attention-neural-dependency/)
18. Timothy Dozat, Peng Qi, and Christopher D. Manning. 2017. Stanford’s graph-based neural dependency parser at the CoNLL 2017 Shared Task. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 20–30. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/dozat-2017-stanford-s-graph-based-neural/)
19. Edouard Grave, Armand Joulin, and Quentin Berthet. 2018b. Unsupervised alignment of embeddings with Wasserstein Procrustes. CoRR, abs/1805.11222. — [arXiv:1805.11222](https://arxiv.org/abs/1805.11222) [find](https://scholar.google.com/scholar?q=Edouard+Grave%2C+Armand+Joulin%2C+and+Quentin+Berthet%2E+2018b%2E+Unsupervised+alignment+of+embeddings+with+Wasserstein+Procrustes%2E+CoRR%2C+abs%2F1805%2E11222%2E+%E2%80%94+%5BarXiv%3A1805%2E11222%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1805%2E11222%29)
20. Jiang Guo, Wanxiang Che, David Yarowsky, Haifeng Wang, and Ting Liu. 2015. Cross-lingual dependency parsing based on distributed representations. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language… [our card](https://md.research.elamur.ai/free/en/papers/guo-2015-cross-lingual-dependency-parsing-based/)
21. Jiang Guo, Wanxiang Che, David Yarowsky, Haifeng Wang, and Ting Liu. 2016. A representation learning framework for multi-source transfer parsing. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pages 2734–2740. [find](https://scholar.google.com/scholar?q=Jiang+Guo%2C+Wanxiang+Che%2C+David+Yarowsky%2C+Haifeng+Wang%2C+and+Ting+Liu%2E+2016%2E+A+representation+learning+framework+for+multi%2Dsource+transfer+parsing%2E+In+Proceedings+of+the+Thirtieth+AAAI+Conference+on+Art)
22. Yedid Hoshen and Lior Wolf. 2018. Non-adversarial unsupervised word translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 469–478. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yedid+Hoshen+and+Lior+Wolf%2E+2018%2E+Non%2Dadversarial+unsupervised+word+translation%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+469%E2%80%93478%2E+Association+f)
23. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jeremy+Howard+and+Sebastian+Ruder%2E+2018%2E+Universal+language+model+fine%2Dtuning+for+text+classification%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volum)
24. Long Duong, Trevor Cohn, Steven Bird, and Paul Cook. 2015. Cross-lingual transfer for unsupervised dependency parsing without parallel data. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning, pages 113–122. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Long+Duong%2C+Trevor+Cohn%2C+Steven+Bird%2C+and+Paul+Cook%2E+2015%2E+Cross%2Dlingual+transfer+for+unsupervised+dependency+parsing+without+parallel+data%2E+In+Proceedings+of+the+Nineteenth+Conference+on+Computationa)
25. Chao Jiang, Hsiang-Fu Yu, Cho-Jui Hsieh, and Kai-Wei Chang. 2018. Learning word embeddings for low-resource languages by PU learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pape… [find](https://scholar.google.com/scholar?q=Chao+Jiang%2C+Hsiang%2DFu+Yu%2C+Cho%2DJui+Hsieh%2C+and+Kai%2DWei+Chang%2E+2018%2E+Learning+word+embeddings+for+low%2Dresource+languages+by+PU+learning%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapte)
26. Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. A simple, fast, and effective reparameterization of IBM model 2. In Proceedings of the 2013 [find](https://scholar.google.com/scholar?q=Chris+Dyer%2C+Victor+Chahuneau%2C+and+Noah+A%2E+Smith%2E+2013%2E+A+simple%2C+fast%2C+and+effective+reparameterization+of+IBM+model+2%2E+In+Proceedings+of+the+2013)
27. Armand Joulin, Piotr Bojanowski, Tomas Mikolov, Hervé Jégou, and Edouard Grave. 2018. Loss in translation: Learning bilingual word mapping with a retrieval criterion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2979–2984. Association for Computa… [our card](https://md.research.elamur.ai/free/en/papers/joulin-2018-loss-translation-learning-bilingual-word/)
28. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional LSTM feature representations. Transactions of the Association for Computational Linguistics, 4:313–327. [our card](https://md.research.elamur.ai/free/en/papers/kiperwasser-2016-simple-accurate-dependency-parsing-bidirectional/)
29. Ryan McDonald, Slav Petrov, and Keith Hall. 2011. Multi-source transfer of delexicalized dependency parsers. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 62–72. [our card](https://md.research.elamur.ai/free/en/papers/mcdonald-2011-multi-source-transfer-delexicalized-dependency/)
30. Tomas Mikolov, Quoc V. Le, and Ilya Sutskever. 2013. Exploiting similarities among languages for machine translation. CoRR, abs/1309.4168. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Quoc+V%2E+Le%2C+and+Ilya+Sutskever%2E+2013%2E+Exploiting+similarities+among+languages+for+machine+translation%2E+CoRR%2C+abs%2F1309%2E4168%2E)
31. Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2018. Unsupervised machine translation using monolingual corpora only. In International Conference on Learning Representations. [our card](https://md.research.elamur.ai/free/en/papers/lample-2017-unsupervised-machine-translation-using-monolingual/)
32. Tahira Naseem, Regina Barzilay, and Amir Globerson. 2012. Selective sharing for multilingual dependency parsing. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 629–637. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/naseem-2012-selective-sharing-multilingual-dependency-parsing/)
33. Angeliki Lazaridou, Georgiana Dinu, and Marco Baroni. 2015. Hubness and pollution: Delving into cross-space mapping for zero-shot learning. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proces… [find](https://scholar.google.com/scholar?q=Angeliki+Lazaridou%2C+Georgiana+Dinu%2C+and+Marco+Baroni%2E+2015%2E+Hubness+and+pollution%3A+Delving+into+cross%2Dspace+mapping+for+zero%2Dshot+learning%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association)
34. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… [our card](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/)
35. Miryam de Lhoneux, Johannes Bjerva, Isabelle Augenstein, and Anders Søgaard. 2018. Parameter sharing between dependency parsers for related languages. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4992–4997. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Miryam+de+Lhoneux%2C+Johannes+Bjerva%2C+Isabelle+Augenstein%2C+and+Anders+S%C3%B8gaard%2E+2018%2E+Parameter+sharing+between+dependency+parsers+for+related+languages%2E+In+Proceedings+of+the+2018+Conference+on+Empirica)
36. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2012. A universal part-of-speech tagset. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC-2012). [our card](https://md.research.elamur.ai/free/en/papers/petrov-2012-universal-part-speech-tagset/)
37. Ying Lin, Shengqi Yang, Veselin Stoyanov, and Heng Ji. 2018. A multi-lingual multi-task architecture for low-resource sequence labeling. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 799–809. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Ying+Lin%2C+Shengqi+Yang%2C+Veselin+Stoyanov%2C+and+Heng+Ji%2E+2018%2E+A+multi%2Dlingual+multi%2Dtask+architecture+for+low%2Dresource+sequence+labeling%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+fo)
38. Pierre Lison and Jrg Tiedemann. 2016. Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France. European Language Resources Association (ELRA). [our card](https://md.research.elamur.ai/free/en/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/)
39. Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019. Linguistic knowledge and transferability of contextual representations. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… [our card](https://md.research.elamur.ai/free/en/papers/liu-2019-linguistic-knowledge-transferability-contextual-representations/)
40. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 6294– 6305.… [our card](https://md.research.elamur.ai/free/en/papers/mccann-2017-learned-translation-contextualized-word-vectors/)
41. Ye Qi, Devendra Sachan, Matthieu Felix, Sarguna Padmanabhan, and Graham Neubig. 2018. When and why are pre-trained word embeddings useful for neural machine translation? In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua… [our card](https://md.research.elamur.ai/free/en/papers/qi-2018-when-why-are-pre-trained/)
42. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+and+Ilya+Sutskever%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
43. Rudolf Rosa and David Mareček. 2018. CUNI x-ling: Parsing under-resourced languages in CoNLL 2018 UD Shared Task. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 187–196. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rudolf+Rosa+and+David+Marec%CC%8Cek%2E+2018%2E+CUNI+x%2Dling%3A+Parsing+under%2Dresourced+languages+in+CoNLL+2018+UD+Shared+Task%2E+In+Proceedings+of+the+CoNLL+2018+Shared+Task%3A+Multilingual+Parsing+from+Raw+Text+to+U)
44. Aaron Smith, Bernd Bohnet, Miryam de Lhoneux, Joakim Nivre, Yan Shao, and Sara Stymne. 2018a. 82 treebanks, 34 models: Universal dependency parsing with multi-treebank models. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 113–123. A… [find](https://scholar.google.com/scholar?q=Aaron+Smith%2C+Bernd+Bohnet%2C+Miryam+de+Lhoneux%2C+Joakim+Nivre%2C+Yan+Shao%2C+and+Sara+Stymne%2E+2018a%2E+82+treebanks%2C+34+models%3A+Universal+dependency+parsing+with+multi%2Dtreebank+models%2E+In+Proceedings+of+the+Co)
45. Aaron Smith, Miryam de Lhoneux, Sara Stymne, and Joakim Nivre. 2018b. An investigation of the interactions between pre-trained word embeddings, character models and pos tags in dependency parsing. [find](https://scholar.google.com/scholar?q=Aaron+Smith%2C+Miryam+de+Lhoneux%2C+Sara+Stymne%2C+and+Joakim+Nivre%2E+2018b%2E+An+investigation+of+the+interactions+between+pre%2Dtrained+word+embeddings%2C+character+models+and+pos+tags+in+dependency+parsing%2E)
46. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2711–2720. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2711%E2%80%932720%2E)
47. Samuel L Smith, David HP Turban, Steven Hamblin, and Nils Y Hammerla. 2017. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. In International Conference on Learning Representations. [our card](https://md.research.elamur.ai/free/en/papers/smith-2017-offline-bilingual-word-vectors-orthogonal/)
48. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929–1958. [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2014%2E+Dropout%3A+A+simple+way+to+prevent+neural+networks+from+overfitting%2E+Journal+of+Machine+Learning+Rese)
49. Milan Straka and Jana Straková. 2017. Tokenizing, POS tagging, lemmatizing and parsing UD 2.0 with UDPipe. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 88–99, Vancouver, Canada. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/straka-2017-tokenizing-pos-tagging-lemmatizing-parsing/)
50. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC-2012). European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+OPUS%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%2D2012%29%2E+European+Language+Resources+)
51. Jörg Tiedemann. 2015. Cross-lingual dependency parsing with universal dependencies and predicted POS labels. In Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015), pages 340–349. [our card](https://md.research.elamur.ai/free/en/papers/tiedemann-2015-cross-lingual-dependency-parsing-universal/)
52. Jörg Tiedemann. 2017. Cross-lingual dependency parsing for closely related languages - helsinki’s submission to vardial 2017. In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 131–136. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2017%2E+Cross%2Dlingual+dependency+parsing+for+closely+related+languages+%2D+helsinki%E2%80%99s+submission+to+vardial+2017%2E+In+Proceedings+of+the+Fourth+Workshop+on+NLP+for+Similar+Languages%2C+Varie)
53. Dingquan Wang and Jason Eisner. 2016. The galactic dependencies treebanks: Getting more data by synthesizing new languages. Transactions of the Association for Computational Linguistics, 4:491–505. [our card](https://md.research.elamur.ai/free/en/papers/wang-2016-galactic-dependencies-treebanks-getting-more/)
54. Wenhui Wang, Baobao Chang, and Mairgup Mansur. 2018. Improved dependency parsing using implicit word connections learned from unlabeled data. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2857–2863. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wenhui+Wang%2C+Baobao+Chang%2C+and+Mairgup+Mansur%2E+2018%2E+Improved+dependency+parsing+using+implicit+word+connections+learned+from+unlabeled+data%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods)
55. Min Xiao and Yuhong Guo. 2014. Distributed word representation learning for cross-lingual dependency parsing. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, pages 119–129. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Min+Xiao+and+Yuhong+Guo%2E+2014%2E+Distributed+word+representation+learning+for+cross%2Dlingual+dependency+parsing%2E+In+Proceedings+of+the+Eighteenth+Conference+on+Computational+Natural+Language+Learning%2C+pa)
56. Min Xiao and Yuhong Guo. 2015. Annotation projection-based representation learning for cross-lingual dependency parsing. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning, pages 73–82. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Min+Xiao+and+Yuhong+Guo%2E+2015%2E+Annotation+projection%2Dbased+representation+learning+for+cross%2Dlingual+dependency+parsing%2E+In+Proceedings+of+the+Nineteenth+Conference+on+Computational+Natural+Language+L)
57. Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. 2015. Normalized word embedding and orthogonal transform for bilingual word translation. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1006–1011. A… [our card](https://md.research.elamur.ai/free/en/papers/xing-2015-normalized-word-embedding-orthogonal-transform/)
58. Wei Yang, Wei Lu, and Vincent Zheng. 2017. A simple regularization-based algorithm for learning cross-domain word embeddings. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2898–2904. [find](https://scholar.google.com/scholar?q=Wei+Yang%2C+Wei+Lu%2C+and+Vincent+Zheng%2E+2017%2E+A+simple+regularization%2Dbased+algorithm+for+learning+cross%2Ddomain+word+embeddings%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Lang)
59. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. [find](https://scholar.google.com/scholar?q=Daniel+Zeman%2C+Jan+Hajic%CC%8C%2C+Martin+Popel%2C+Martin+Potthast%2C+Milan+Straka%2C+Filip+Ginter%2C+Joakim+Nivre%2C+and+Slav+Petrov%2E+2018%2E)
60. CoNLL 2018 Shared Task: Multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–21, Brussels, Belgium. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/zeman-2018-conll-2018-shared-task-multilingual/)
61. Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Adversarial training for unsupervised bilingual lexicon induction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1959–1970. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/zhang-2017-adversarial-training-unsupervised-bilingual-lexicon/)
62. Yuan Zhang and Regina Barzilay. 2015. Hierarchical low-rank tensors for multilingual transfer parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1857–1867. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yuan+Zhang+and+Regina+Barzilay%2E+2015%2E+Hierarchical+low%2Drank+tensors+for+multilingual+transfer+parsing%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+)