---
title: "Learning Language Representations for Typology Prediction (2017)"
description: "The authors test whether a neural model trained on parallel text can fill these gaps. 42 references, cited by 67 papers here."
authors: ["Chaitanya Malaviya", "Graham Neubig", "Patrick Littell"]
year: 2017
venue: "Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D17-1268/"
doi: "10.18653/v1/d17-1268"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/malaviya-2017-learning-language-representations-typology-prediction/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/malaviya-2017-learning-language-representations-typology-prediction/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) / Learning Language Representations for Typology Prediction

# Learning Language Representations for Typology Prediction

**Authors:** [Chaitanya Malaviya](https://md.research.elamur.ai/free/en/authors/chaitanya-malaviya/), [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/) and [Patrick Littell](https://md.research.elamur.ai/free/en/authors/patrick-littell/) · **Year:** 2017 · **Venue:** [Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2017](https://md.research.elamur.ai/free/en/venues/emnlp/2017/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D17-1268/) · [PDF](https://aclanthology.org/D17-1268.pdf) · [DOI 10.18653/v1/d17-1268](https://doi.org/10.18653/v1/d17-1268) · [OpenAlex](https://openalex.org/W2740900542)

**Citations:** 120 citations (OpenAlex, 2026-10-14) · [67](https://md.research.elamur.ai/free/en/papers/malaviya-2017-learning-language-representations-typology-prediction/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Typological databases such as WALS are sparsely filled, while NLP applications often need complete feature sets for the languages they handle. The authors test whether a neural model trained on parallel text can fill these gaps. They train a many-to-one NMT system from 1017 languages into English on Bible translations scraped from bible.com, then derive per-language vectors in three ways: a language-token embedding from a multilingual LSTM LM (LMVEC), the same kind of embedding from the NMT model (MTVEC), and a new representation that averages the encoder's LSTM cell states over all of a language's sentences (MTCELL). Per-feature logistic regression classifiers use these vectors to predict binary URIEL features covering syntax, phonology, and phonetic inventory. They are evaluated with and without a geographic/genetic 3-NN baseline. Combining both MT representations (MTBOTH) works best, beating both the majority-class rate and the neighbor baseline. The main gains are in syntactic features such as word order and case marking. This suggests that massively multilingual translation models encode recoverable typological knowledge that can help document under-described languages.

## Key points
- Data: a Bible corpus covering 1017 languages and about 20.6M training sentences. Vocabulary is shared through joint BPE with 32K merges. The model is a single-layer 512-dim LSTM built in DyNet, trained with Adam (lr 0.001) and dropout 0.5.
- Targets: URIEL binary features, namely 103 syntax, 28 phonology, and 158 phonetic inventory features. Evaluation uses 10-fold cross-validation over languages.
- Syntax accuracy without auxiliary features rises from 69.91 (majority class) to 77.11 with MTBOTH. With 3-NN neighbor features added, it goes from 83.07 to 86.33.
- MTVEC beats LMVEC by roughly 2–3 points without auxiliary information, and MTCELL beats MTVEC. A paired bootstrap test finds the gains over the neighbor baseline significant for syntax and inventory but not for phonology.
- The largest single gain is for numeral-after-noun word order (37.40 → 81.26). Some features, such as gender markers, case suffix, and negative affix, got slightly worse.

## Interesting ideas
- **Averaging an encoder's internal states to describe a whole input set.** Instead of relying on a learned ID embedding, the method pools cell states across every token of every sentence. This kind of dynamic profile could characterize dialects, domains, or authors in other projects.
- **Using cell states rather than hidden states.** The authors found the gated hidden outputs carried less variance and information than the raw cell memory. This is a useful default when probing LSTMs.
- **Translation alignment as supervision for representations.** Translating into one shared target language gives a stronger signal than plain language modeling. With the geographic/genetic baseline added, the LM vectors' advantage disappeared while the MT vectors still helped. Parallel data in a pivot language such as English could similarly sharpen representations of Ukrainian and related low-resource varieties.
- **Testing against a strong "neighbors" baseline.** The comparison includes a geographic/genetic k-NN baseline and also combinations with it. This shows whether learned vectors add anything beyond simply knowing where a language is spoken, which is a good control for any geography-correlated prediction task.
- **Checking where surprising gains come from.** The unexpected improvements on phonology were traced to regionally clustered features, suggesting the model detects regions through syntax. Examining such side effects guards against over-interpreting what a probe "knows."

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 365 works · 158,048 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: Chaitanya Malaviya, Graham Neubig and Patrick Littell. 2017. Learning Language Representations for Typology Prediction. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{malaviya2017learning,
  title = {{Learning Language Representations for Typology Prediction}},
  author = {Chaitanya Malaviya, Graham Neubig and Patrick Littell},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1268},
  url = {https://doi.org/10.18653/v1/d17-1268}
}
```

- [← previous A Dataset for Multi-Target Stance Detection](https://md.research.elamur.ai/free/en/papers/sobhani-2017-dataset-multi-target-stance-detection/)
- [next → Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings](https://md.research.elamur.ai/free/en/papers/gonzales-2017-improving-word-sense-disambiguation-neural/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/)

## References

42 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah Smith. 2016. [Many languages, one parser](https://md.research.elamur.ai/free/en/papers/ammar-2016-many-languages-one-parser/). Transactions of the Association for Computational Linguistics, 4:431–444.
2. Gregory D.S. Anderson. 2013. The velar nasal. In Matthew S. Dryer and Martin Haspelmath, editors, The World Atlas of Language Structures Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. [find](https://scholar.google.com/scholar?q=Gregory+D%2ES%2E+Anderson%2E+2013%2E+The+velar+nasal%2E+In+Matthew+S%2E+Dryer+and+Martin+Haspelmath%2C+editors%2C+The+World+Atlas+of+Language+Structures+Online%2E+Max+Planck+Institute+for+Evolutionary+Anthropology%2C+Lei)
3. Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. 2017. [What do neural machine translation models learn about morphology?](https://md.research.elamur.ai/free/en/papers/belinkov-2017-what-do-neural-machine-translation/) In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.
4. Yevgeni Berzak, Roi Reichart, and Boris Katz. 2014. Reconstructing native language typology from foreign language usage. In Eighteenth Conference on Computational Natural Language Learning (CoNLL). [find](https://scholar.google.com/scholar?q=Yevgeni+Berzak%2C+Roi+Reichart%2C+and+Boris+Katz%2E+2014%2E+Reconstructing+native+language+typology+from+foreign+language+usage%2E+In+Eighteenth+Conference+on+Computational+Natural+Language+Learning+%28CoNLL%29%2E)
5. Ranko Bugarski. 1991. Contrastive analysis of terminology and the terminology of contrastive analysis. Languages in Contact und Contrast. Essays in Contact Linguistics/Edited by Vladimir Ivir and Damir Kalogjera.–Berlin, pages 73–82. [find](https://scholar.google.com/scholar?q=Ranko+Bugarski%2E+1991%2E+Contrastive+analysis+of+terminology+and+the+terminology+of+contrastive+analysis%2E+Languages+in+Contact+und+Contrast%2E+Essays+in+Contact+Linguistics%2FEdited+by+Vladimir+Ivir+and+Dami)
6. Reed Coke, Ben King, and Dragomir Radev. 2016. Classifying syntactic regularities for hundreds of languages. arXiv preprint arXiv:1603.08016. — [arXiv:1603.08016](https://arxiv.org/abs/1603.08016) [find](https://scholar.google.com/scholar?q=Reed+Coke%2C+Ben+King%2C+and+Dragomir+Radev%2E+2016%2E+Classifying+syntactic+regularities+for+hundreds+of+languages%2E+arXiv+preprint+arXiv%3A1603%2E08016%2E+%E2%80%94+%5BarXiv%3A1603%2E08016%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1603%2E08016%29)
7. Chris Collins and Richard Kayne. 2011. Syntactic Structures of the World’s Languages. New York University, New York. [find](https://scholar.google.com/scholar?q=Chris+Collins+and+Richard+Kayne%2E+2011%2E+Syntactic+Structures+of+the+World%E2%80%99s+Languages%2E+New+York+University%2C+New+York%2E)
8. Bernard Comrie. 1989. Language Universals and Linguistic Typology: Syntax and Morphology. Blackwell, Oxford. [find](https://scholar.google.com/scholar?q=Bernard+Comrie%2E+1989%2E+Language+Universals+and+Linguistic+Typology%3A+Syntax+and+Morphology%2E+Blackwell%2C+Oxford%2E)
9. Joachim Daiber, Miloš Stanojević, and Khalil Sima’an. 2016. Universal reordering via linguistic typology. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3167–3176, Osaka, Japan. [find](https://scholar.google.com/scholar?q=Joachim+Daiber%2C+Milos%CC%8C+Stanojevic%CC%81%2C+and+Khalil+Sima%E2%80%99an%2E+2016%2E+Universal+reordering+via+linguistic+typology%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conference+on+Computational+Linguistic)
10. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
11. Hal Daumé III. 2009. Non-parametric Bayesian areal linguistics. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 593–601, Boulder, Colorado. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hal+Daume%CC%81+III%2E+2009%2E+Non%2Dparametric+Bayesian+areal+linguistics%2E+In+Proceedings+of+Human+Language+Technologies%3A+The+2009+Annual+Conference+of+the+North+American+Chapter+of+the+Association+for+Computat)
12. Hal Daumé III and Lyle Campbell. 2007. A Bayesian model for discovering typological implications. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, pages 65–72, Prague, Czech Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hal+Daume%CC%81+III+and+Lyle+Campbell%2E+2007%2E+A+Bayesian+model+for+discovering+typological+implications%2E+In+Proceedings+of+the+45th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+65%E2%80%93)
13. Matthew S. Dryer and Martin Haspelmath, editors. 2013. WALS Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. [find](https://scholar.google.com/scholar?q=Matthew+S%2E+Dryer+and+Martin+Haspelmath%2C+editors%2E+2013%2E+WALS+Online%2E+Max+Planck+Institute+for+Evolutionary+Anthropology%2C+Leipzig%2E)
14. Noelia Ramón Garcı́a. 2002. Contrastive linguistics and translation studies interconnected: The corpus-based approach. Linguistica Antverpiensia, New Series–Themes in Translation Studies, (1). [find](https://scholar.google.com/scholar?q=Noelia+Ramo%CC%81n+Garc%C4%B1%CC%81a%2E+2002%2E+Contrastive+linguistics+and+translation+studies+interconnected%3A+The+corpus%2Dbased+approach%2E+Linguistica+Antverpiensia%2C+New+Series%E2%80%93Themes+in+Translation+Studies%2C+%281%29%2E)
15. Joseph Greenberg. 1963. Some universals of grammar with particular reference to the order of meaningful elements. In Joseph Greenberg, editor, Universals of Language, pages 110–113. MIT Press, London. [find](https://scholar.google.com/scholar?q=Joseph+Greenberg%2E+1963%2E+Some+universals+of+grammar+with+particular+reference+to+the+order+of+meaningful+elements%2E+In+Joseph+Greenberg%2C+editor%2C+Universals+of+Language%2C+pages+110%E2%80%93113%2E+MIT+Press%2C+London%2E)
16. Thanh-Le Ha, Jan Niehues, and Alexander Waibel. 2016. [Toward multilingual neural machine translation with universal encoder and decoder](https://md.research.elamur.ai/free/en/papers/ha-2016-multilingual-neural-machine-translation-universal/). arXiv preprint arXiv:1611.04798.
17. Harald Hammarström, Robert Forkel, Martin Haspelmath, and Sebastian Bank. 2015. Glottolog 2.6. Max Planck Institute for the Science of Human History, Jena. [find](https://scholar.google.com/scholar?q=Harald+Hammarstro%CC%88m%2C+Robert+Forkel%2C+Martin+Haspelmath%2C+and+Sebastian+Bank%2E+2015%2E+Glottolog+2%2E6%2E+Max+Planck+Institute+for+the+Science+of+Human+History%2C+Jena%2E)
18. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%931780%2E)
19. Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2016. Google’s multilingual neural machine translation system: Enabling zero-shot translation. arXiv preprint arXiv:1611.04558. [find](https://scholar.google.com/scholar?q=Melvin+Johnson%2C+Mike+Schuster%2C+Quoc+V+Le%2C+Maxim+Krikun%2C+Yonghui+Wu%2C+Zhifeng+Chen%2C+Nikhil+Thorat%2C+Fernanda+Vie%CC%81gas%2C+Martin+Wattenberg%2C+Greg+Corrado%2C+et+al%2E+2016%2E+Google%E2%80%99s+multilingual+neural+machine+tr)
20. Andrej Karpathy, Justin Johnson, and Li Fei-Fei. 2015. [Visualizing and understanding recurrent networks](https://md.research.elamur.ai/free/en/papers/karpathy-2015-visualizing-understanding-recurrent-networks/). arXiv preprint arXiv:1506.02078.
21. Adhiguna Kuncoro, Miguel Ballesteros, Lingpeng Kong, Chris Dyer, Graham Neubig, and Noah A. Smith. 2017. What do recurrent neural network grammars learn about syntax? In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers… [find](https://scholar.google.com/scholar?q=Adhiguna+Kuncoro%2C+Miguel+Ballesteros%2C+Lingpeng+Kong%2C+Chris+Dyer%2C+Graham+Neubig%2C+and+Noah+A%2E+Smith%2E+2017%2E+What+do+recurrent+neural+network+grammars+learn+about+syntax%3F+In+Proceedings+of+the+15th+Confer)
22. M. Paul Lewis, Gary F. Simons, and Charles D. Fennig. 2015. Ethnologue: Languages of the World, Eighteenth edition. SIL International, Dallas, Texas. [find](https://scholar.google.com/scholar?q=M%2E+Paul+Lewis%2C+Gary+F%2E+Simons%2C+and+Charles+D%2E+Fennig%2E+2015%2E+Ethnologue%3A+Languages+of+the+World%2C+Eighteenth+edition%2E+SIL+International%2C+Dallas%2C+Texas%2E)
23. William D Lewis and Fei Xia. 2008. Automatically identifying computationally relevant typological features. In Proceedings of the Third International Joint Conference on Natural Language Processing, Volume II, pages 685–690. [find](https://scholar.google.com/scholar?q=William+D+Lewis+and+Fei+Xia%2E+2008%2E+Automatically+identifying+computationally+relevant+typological+features%2E+In+Proceedings+of+the+Third+International+Joint+Conference+on+Natural+Language+Processing%2C+V)
24. Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. [Uriel and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors](https://md.research.elamur.ai/free/en/papers/littell-2017-uriel-lang2vec-representing-languages-as/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational…
25. Ian Maddieson. 2013a. Lateral consonants. In Matthew S. Dryer and Martin Haspelmath, editors, The World Atlas of Language Structures Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. [find](https://scholar.google.com/scholar?q=Ian+Maddieson%2E+2013a%2E+Lateral+consonants%2E+In+Matthew+S%2E+Dryer+and+Martin+Haspelmath%2C+editors%2C+The+World+Atlas+of+Language+Structures+Online%2E+Max+Planck+Institute+for+Evolutionary+Anthropology%2C+Leipzig)
26. Ian Maddieson. 2013b. Uvular consonants. In Matthew S. Dryer and Martin Haspelmath, editors, The World Atlas of Language Structures Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. [find](https://scholar.google.com/scholar?q=Ian+Maddieson%2E+2013b%2E+Uvular+consonants%2E+In+Matthew+S%2E+Dryer+and+Martin+Haspelmath%2C+editors%2C+The+World+Atlas+of+Language+Structures+Online%2E+Max+Planck+Institute+for+Evolutionary+Anthropology%2C+Leipzig%2E)
27. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Interspeech, volume 2, page 3. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Martin+Karafia%CC%81t%2C+Lukas+Burget%2C+Jan+Cernocky%CC%80%2C+and+Sanjeev+Khudanpur%2E+2010%2E+Recurrent+neural+network+based+language+model%2E+In+Interspeech%2C+volume+2%2C+page+3%2E)
28. Steven Moran, Daniel McCloy, and Richard Wright. 2014. PHOIBLE Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. [find](https://scholar.google.com/scholar?q=Steven+Moran%2C+Daniel+McCloy%2C+and+Richard+Wright%2E+2014%2E+PHOIBLE+Online%2E+Max+Planck+Institute+for+Evolutionary+Anthropology%2C+Leipzig%2E)
29. Tahira Naseem, Regina Barzilay, and Amir Globerson. 2012. [Selective sharing for multilingual dependency parsing](https://md.research.elamur.ai/free/en/papers/naseem-2012-selective-sharing-multilingual-dependency-parsing/). In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 629–637. Association for Computational Linguistics.
30. Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, et al. 2017. [Dynet: The dynamic neural network toolkit](https://md.research.elamur.ai/free/en/papers/neubig-2017-dynet-dynamic-neural-network-toolkit/). arXiv preprint arXiv:1701.03980.
31. Joanna Nichols. 1992. Linguistic Diversity in Space and Time. University of Chicago Press, Chicago. [find](https://scholar.google.com/scholar?q=Joanna+Nichols%2E+1992%2E+Linguistic+Diversity+in+Space+and+Time%2E+University+of+Chicago+Press%2C+Chicago%2E)
32. Helen O’Horan, Yevgeni Berzak, Ivan Vulic, Roi Reichart, and Anna Korhonen. 2016. Survey on the use of typological information in natural language processing. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1297–1308, Osaka, Japa… [find](https://scholar.google.com/scholar?q=Helen+O%E2%80%99Horan%2C+Yevgeni+Berzak%2C+Ivan+Vulic%2C+Roi+Reichart%2C+and+Anna+Korhonen%2E+2016%2E+Survey+on+the+use+of+typological+information+in+natural+language+processing%2E+In+Proceedings+of+COLING+2016%2C+the+26th+I)
33. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
34. Robert Östling. 2015. Word order typology through multilingual word alignment. In The 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 205–211. [find](https://scholar.google.com/scholar?q=Robert+O%CC%88stling%2E+2015%2E+Word+order+typology+through+multilingual+word+alignment%2E+In+The+53rd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+7th+International+Joint+Conference+o)
35. Robert Östling and Jörg Tiedemann. 2017. Continuous multilinguality with language vectors. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 644–649, Valencia, Spain. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Robert+O%CC%88stling+and+Jo%CC%88rg+Tiedemann%2E+2017%2E+Continuous+multilinguality+with+language+vectors%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguist)
36. Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. 2017. [Learning to generate reviews and discovering sentiment](https://md.research.elamur.ai/free/en/papers/radford-2017-learning-generate-reviews-discovering-sentiment/). arXiv preprint arXiv:1704.01444.
37. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Li…
38. Xing Shi, Inkit Padhi, and Kevin Knight. 2016. [Does string-based neural MT learn source syntax?](https://md.research.elamur.ai/free/en/papers/shi-2016-does-string-based-neural-mt/) In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1526– 1534, Austin, Texas. Association for Computational Linguistics.
39. Oscar Täckström, Ryan McDonald, and Joakim Nivre. 2013. [Target language adaptation of discriminative transfer parsers](https://md.research.elamur.ai/free/en/papers/tackstrom-2013-target-language-adaptation-discriminative-transfer/). In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1061–1071, Atlanta, Georgi…
40. Hiroya Takamura, Ryo Nagata, and Yoshifumi Kawasaki. 2016. Discriminative analysis of linguistic features for typological study. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Hiroya+Takamura%2C+Ryo+Nagata%2C+and+Yoshifumi+Kawasaki%2E+2016%2E+Discriminative+analysis+of+linguistic+features+for+typological+study%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resour)
41. Yulia Tsvetkov, Sunayana Sitaram, Manaal Faruqui, Guillaume Lample, Patrick Littell, David Mortensen, Alan W Black, Lori Levin, and Chris Dyer. 2016. Polyglot neural language models: A case study in cross-lingual phonetic representation learning. In Proceedings of the 2016 Conference of the North Am… [find](https://scholar.google.com/scholar?q=Yulia+Tsvetkov%2C+Sunayana+Sitaram%2C+Manaal+Faruqui%2C+Guillaume+Lample%2C+Patrick+Littell%2C+David+Mortensen%2C+Alan+W+Black%2C+Lori+Levin%2C+and+Chris+Dyer%2E+2016%2E+Polyglot+neural+language+models%3A+A+case+study+in+c)
42. Yuan Zhang, Roi Reichart, Regina Barzilay, and Amir Globerson. 2012. Learning to map into a universal POS tagset. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1368–1378. Association for Computationa… [find](https://scholar.google.com/scholar?q=Yuan+Zhang%2C+Roi+Reichart%2C+Regina+Barzilay%2C+and+Amir+Globerson%2E+2012%2E+Learning+to+map+into+a+universal+POS+tagset%2E+In+Proceedings+of+the+2012+Joint+Conference+on+Empirical+Methods+in+Natural+Language+P)

*Updated 24 October 2026*