---
title: "AmericasNLI: Evaluating Zero-shot Natural — Ebrahimi 2022"
description: "Most evaluations of multilingual models on languages missing from their pretraining data use syntactic tasks such as POS tagging, parsing or NER."
authors: ["Abteen Ebrahimi", "Manuel Mager", "Arturo Oncevay", "Vishrav Chaudhary", "Luis Chiruzzo", "Angela Fan", "John Ortega", "Ricardo Ramos", "Annette Rios", "Ivan Vladimir Meza Ruiz", "Gustavo Giménez-Lugo", "Elisabeth Mager", "Graham Neubig", "Alexis Palmer", "Rolando Coto-Solano", "Thang Vu", "Katharina Kann"]
year: 2022
venue: "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2022.acl-long.435/"
doi: "10.18653/v1/2022.acl-long.435"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ebrahimi-2022-americasnli-evaluating-zero-shot-natural/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ebrahimi-2022-americasnli-evaluating-zero-shot-natural/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/) / AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages

# AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages

**Authors:** [Abteen Ebrahimi](https://md.research.elamur.ai/free/en/authors/abteen-ebrahimi/), [Manuel Mager](https://md.research.elamur.ai/free/en/authors/manuel-mager/), [Arturo Oncevay](https://md.research.elamur.ai/free/en/authors/arturo-oncevay/), [Vishrav Chaudhary](https://md.research.elamur.ai/free/en/authors/vishrav-chaudhary/), [Luis Chiruzzo](https://md.research.elamur.ai/free/en/authors/luis-chiruzzo/), [Angela Fan](https://md.research.elamur.ai/free/en/authors/angela-fan/), [John Ortega](https://md.research.elamur.ai/free/en/authors/john-ortega/), [Ricardo Ramos](https://md.research.elamur.ai/free/en/authors/ricardo-ramos/), [Annette Rios](https://md.research.elamur.ai/free/en/authors/annette-rios/), [Ivan Vladimir Meza Ruiz](https://md.research.elamur.ai/free/en/authors/ivan-vladimir-meza-ruiz/), [Gustavo Giménez-Lugo](https://md.research.elamur.ai/free/en/authors/gustavo-gimenez-lugo/), Elisabeth Mager et al. (17 authors) · **Year:** 2022 · **Venue:** Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.acl-long.435/) · [PDF](https://aclanthology.org/2022.acl-long.435.pdf) · [DOI 10.18653/v1/2022.acl-long.435](https://doi.org/10.18653/v1/2022.acl-long.435) · [OpenAlex](https://openalex.org/W3154311556)

**Citations:** 66 citations (openalex-snapshot · 2026-10-14) · [97](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary

Most evaluations of multilingual models on languages missing from their pretraining data use syntactic tasks such as POS tagging, parsing or NER. It was unclear whether zero-shot transfer also works for semantic understanding. The authors built AmericasNLI, a natural language inference benchmark for 10 Indigenous languages of the Americas. Its examples are human translations of part of the Spanish XNLI, so they stay parallel with the 15 existing XNLI languages. Using XLM-R, they compared plain zero-shot transfer, continued pretraining on target-language monolingual text, and translation-based setups. They also tested for dataset artifacts with hypothesis-only models and had humans check label validity for Bribri.

Out-of-the-box XLM-R beats chance only slightly. Continued pretraining helps. The biggest surprise is that finetuning on machine-translated training data works best, even though the translation systems are poor. The benchmark gives a realistic testbed for truly low-resource languages and makes these communities more visible in NLP.

## Key points

- **Languages covered:** the ten languages are Shipibo-Konibo, Wixarika, Quechua, Nahuatl, Aymara, Rarámuri, Bribri, Otomí, Guaraní and Asháninka. They come from eight language families.
- **Data selection:** up to 750 dev and 750 test pairs per language. The pairs come from XNLI genres judged easier to translate: "face-to-face", "letters" and "telephone".
- **Average XLM-R accuracy:**
  - zero-shot: 38.48%
  - with continued pretraining: 43.85% (up to +5.86 points)
  - translate-train: 49.12% (+11.13 points over the unadapted model)
- **Artifact check:** hypothesis-only models mostly score above chance (translate-train peaks at 44.23%). Full-example models still generally do better, so the models learn more than artifacts.
- **Limitations and availability:** the test sets are small, so the authors include a power analysis. Experiments mainly use a single model family and one adaptation method. The data is released at github.com/abteen/americasnli.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

97 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 92 not yet.

- [MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition](https://md.research.elamur.ai/free/en/papers/adelani-2022-masakhaner-2-0-africa-centric/)
- [IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages](https://md.research.elamur.ai/free/en/papers/gala-2023-indictrans2-towards-high-quality-accessible/)
- [Findings of the AmericasNLP 2021 Shared Task on Open Machine Translation for Indigenous Languages of the Americas](https://md.research.elamur.ai/free/en/papers/mager-2021-findings-americasnlp-2021-shared-task/)
- [Lifting the Curse of Multilinguality by Pre-training Modular Transformers](https://md.research.elamur.ai/free/en/papers/pfeiffer-2022-lifting-curse-multilinguality-pre-training/)
- [Voice Activation Detection for Transcription of Indigenous Languages](https://md.research.elamur.ai/free/en/papers/solano-2026-voice-activation-detection-transcription-indigenous/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ebrahimi2022americasnli,
  title = {{AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages}},
  author = {Ebrahimi, Abteen and Mager, Manuel and Oncevay, Arturo and Chaudhary, Vishrav and Chiruzzo, Luis and Fan, Angela and Ortega, John and Ramos, Ricardo and Rios, Annette and Ruiz, Ivan Vladimir Meza and Giménez-Lugo, Gustavo and Mager, Elisabeth and Neubig, Graham and Palmer, Alexis and Coto-Solano, Rolando and Vu, Thang and Kann, Katharina},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2022},
  doi = {10.18653/v1/2022.acl-long.435},
  url = {https://doi.org/10.18653/v1/2022.acl-long.435}
}
```

- [← previous AlexU-AL at SemEval-2022 Task 6: Detecting Sarcasm in Arabic Text Using Deep Learning Techniques](https://md.research.elamur.ai/free/en/papers/lotfy-2022-alexu-al-semeval-2022-task/)
- [next → An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models](https://md.research.elamur.ai/free/en/papers/meade-2022-empirical-survey-effectiveness-debiasing-techniques/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/)

## References

93 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Željko Agić and Ivan Vulić. 2019. JW300: A wide-coverage parallel corpus for low-resource languages. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3204–3210, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=%C5%BDeljko+Agic%CC%81+and+Ivan+Vulic%CC%81%2E+2019%2E+JW300%3A+A+wide%2Dcoverage+parallel+corpus+for+low%2Dresource+languages%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages) [our card](https://md.research.elamur.ai/free/en/papers/agic-2019-jw300-wide-coverage-parallel-corpus/)
2. Maytham Alabbas. 2013. A dataset for Arabic textual entailment. In Proceedings of the Student Research Workshop associated with RANLP 2013, pages 7– 13, Hissar, Bulgaria. INCOMA Ltd. Shoumen, BUL-GARIA. [find](https://scholar.google.com/scholar?q=Maytham+Alabbas%2E+2013%2E+A+dataset+for+Arabic+textual+entailment%2E+In+Proceedings+of+the+Student+Research+Workshop+associated+with+RANLP+2013%2C+pages+7%E2%80%93+13%2C+Hissar%2C+Bulgaria%2E+INCOMA+Ltd%2E+Shoumen%2C+BUL%2DGARI)
3. Hossein Amirkhani, Mohammad AzariJafari, Azadeh Amirak, Zohreh Pourjafari, Soroush Faridan Jahromi, and Zeinab Kouhkan. 2020. Farstail: A persian natural language inference dataset. ArXiv, abs/2009.08820. — [arXiv:2009.08820](https://arxiv.org/abs/2009.08820) [find](https://scholar.google.com/scholar?q=Hossein+Amirkhani%2C+Mohammad+AzariJafari%2C+Azadeh+Amirak%2C+Zohreh+Pourjafari%2C+Soroush+Faridan+Jahromi%2C+and+Zeinab+Kouhkan%2E+2020%2E+Farstail%3A+A+persian+natural+language+inference+dataset%2E+ArXiv%2C+abs%2F2009%2E08)
4. Waleed Ammar, George Mulcaire, Yulia Tsvetkov, Guillaume Lample, Chris Dyer, and Noah A. Smith. 2016. Massively multilingual word embeddings. ArXiv, abs/1602.01925. [find](https://scholar.google.com/scholar?q=Waleed+Ammar%2C+George+Mulcaire%2C+Yulia+Tsvetkov%2C+Guillaume+Lample%2C+Chris+Dyer%2C+and+Noah+A%2E+Smith%2E+2016%2E+Massively+multilingual+word+embeddings%2E+ArXiv%2C+abs%2F1602%2E01925%2E) [our card](https://md.research.elamur.ai/free/en/papers/ammar-2016-massively-multilingual-word-embeddings/)
5. M. Artetxe and Holger Schwenk. 2019. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. Transactions of the Association for Computational Linguistics, 7:597–610. [find](https://scholar.google.com/scholar?q=M%2E+Artetxe+and+Holger+Schwenk%2E+2019%2E+Massively+multilingual+sentence+embeddings+for+zero%2Dshot+cross%2Dlingual+transfer+and+beyond%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+7%3A597%E2%80%9361) [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2018-massively-multilingual-sentence-embeddings-zero/)
6. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. [find](https://scholar.google.com/scholar?q=Piotr+Bojanowski%2C+Edouard+Grave%2C+Armand+Joulin%2C+and+Tomas+Mikolov%2E+2017%2E+Enriching+word+vectors+with+subword+information%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+5%3A135%E2%80%93146%2E)
7. Johan Bos, Fabio Massimo Zanzotto, and M. Pennacchiotti. 2009. Textual entailment at evalita 2009. [find](https://scholar.google.com/scholar?q=Johan+Bos%2C+Fabio+Massimo+Zanzotto%2C+and+M%2E+Pennacchiotti%2E+2009%2E+Textual+entailment+at+evalita+2009%2E)
8. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+Proceedings+of+the+2015+Conference+on+Empirica)
9. David Brambila. 1976. Diccionario raramuri - castellano: Tarahumar. [find](https://scholar.google.com/scholar?q=David+Brambila%2E+1976%2E+Diccionario+raramuri+%2D+castellano%3A+Tarahumar%2E)
10. Gabriela Caballero. 2008. Choguita rarámuri (tarahumara) phonology and morphology. [find](https://scholar.google.com/scholar?q=Gabriela+Caballero%2E+2008%2E+Choguita+rar%C3%A1muri+%28tarahumara%29+phonology+and+morphology%2E)
11. M. Cajero. 1998. Raíces del Otomí: diccionario. Gobierno del Estado de Tlaxcala. [find](https://scholar.google.com/scholar?q=M%2E+Cajero%2E+1998%2E+Ra%C3%ADces+del+Otom%C3%AD%3A+diccionario%2E+Gobierno+del+Estado+de+Tlaxcala%2E)
12. Mateo Cajero. 2009. Historia de los Otomíes en Ixtenco, volume 1. Instituto Tlaxcalteca de la Cultura, Tlaxcala, México. [find](https://scholar.google.com/scholar?q=Mateo+Cajero%2E+2009%2E+Historia+de+los+Otom%C3%ADes+en+Ixtenco%2C+volume+1%2E+Instituto+Tlaxcalteca+de+la+Cultura%2C+Tlaxcala%2C+M%C3%A9xico%2E)
13. Lyle Campbell. 2000. American Indian languages: the historical linguistics of Native America. Oxford University Press. [find](https://scholar.google.com/scholar?q=Lyle+Campbell%2E+2000%2E+American+Indian+languages%3A+the+historical+linguistics+of+Native+America%2E+Oxford+University+Press%2E)
14. D. Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, and Dan Jurafsky. 2020. With little power comes great responsibility. In EMNLP. [find](https://scholar.google.com/scholar?q=D%2E+Card%2C+Peter+Henderson%2C+Urvashi+Khandelwal%2C+Robin+Jia%2C+Kyle+Mahowald%2C+and+Dan+Jurafsky%2E+2020%2E+With+little+power+comes+great+responsibility%2E+In+EMNLP%2E)
15. Ethan C. Chau, Lucy H. Lin, and Noah A. Smith. 2020. Parsing with multilingual BERT, a small corpus, and a small treebank. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1324–1334, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ethan+C%2E+Chau%2C+Lucy+H%2E+Lin%2C+and+Noah+A%2E+Smith%2E+2020%2E+Parsing+with+multilingual+BERT%2C+a+small+corpus%2C+and+a+small+treebank%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2C+pag)
16. Luis Chiruzzo, Pedro Amarilla, Adolfo Ríos, and Gustavo Giménez Lugo. 2020. Development of a Guarani - Spanish parallel corpus. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 2629–2633, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Luis+Chiruzzo%2C+Pedro+Amarilla%2C+Adolfo+R%C3%ADos%2C+and+Gustavo+Gim%C3%A9nez+Lugo%2E+2020%2E+Development+of+a+Guarani+%2D+Spanish+parallel+corpus%2E+In+Proceedings+of+the+12th+Language+Resources+and+Evaluation+Conference%2C)
17. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, E. Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In ACL. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+E%2E+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlingual+re)
18. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computation… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Ruty+Rinott%2C+Guillaume+Lample%2C+Adina+Williams%2C+Samuel+R%2E+Bowman%2C+Holger+Schwenk%2C+and+Veselin+Stoyanov%2E+2018%2E+Xnli%3A+Evaluating+cross%2Dlingual+sentence+representations%2E+In+Proceedings+of+)
19. Adolfo Constenla, Feliciano Elizondo, and Francisco Pereira. 2004. Curso Básico de Bribri. Editorial de la Universidad de Costa Rica. [find](https://scholar.google.com/scholar?q=Adolfo+Constenla%2C+Feliciano+Elizondo%2C+and+Francisco+Pereira%2E+2004%2E+Curso+B%C3%A1sico+de+Bribri%2E+Editorial+de+la+Universidad+de+Costa+Rica%2E)
20. Rubén Cushimariano Romano and Richer C. Sebastián Q. 2008. Ñaantsipeta asháninkaki bi-rakochaki. diccionario asháninka-castellano. versión preliminar. [http://www.lengamer.org/](http://www.lengamer.org/) publicaciones/diccionarios/. Visitado: 01/03/2013. [↗](http://www.lengamer.org/)
21. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
22. Rafaela Alves Dos Santos. 2017. DIGLOSSIA NO PARAGUAI: A restrição dos monolíngues em guarani no acesso à informação. Trabalho de Conclusão de Curso, Bacharelado em Línguas Es-trangeiras. Universidade de Brasília, Brasilia. [find](https://scholar.google.com/scholar?q=Rafaela+Alves+Dos+Santos%2E+2017%2E+DIGLOSSIA+NO+PARAGUAI%3A+A+restri%C3%A7%C3%A3o+dos+monol%C3%ADngues+em+guarani+no+acesso+%C3%A0+informa%C3%A7%C3%A3o%2E+Trabalho+de+Conclus%C3%A3o+de+Curso%2C+Bacharelado+em+L%C3%ADnguas+Es%2Dtrangeiras%2E+Universidade)
23. Kathrin Eichler, Aleksandra Gabryszak, and Günter Neumann. 2014. An analysis of textual inference in German customer emails. In Proceedings of the Third Joint Conference on Lexical and Computational Semantics (*SEM 2014), pages 69–74, Dublin, Ireland. Association for Computational Linguistics and Du… [find](https://scholar.google.com/scholar?q=Kathrin+Eichler%2C+Aleksandra+Gabryszak%2C+and+G%C3%BCnter+Neumann%2E+2014%2E+An+analysis+of+textual+inference+in+German+customer+emails%2E+In+Proceedings+of+the+Third+Joint+Conference+on+Lexical+and+Computational+S)
24. Norma Faust. 1973. Lecciones para el aprendizaje del idioma shipibo-conibo, volume 1 of Documento de Trabajo. Instituto Lingüístico de Verano, Yarinacocha. [find](https://scholar.google.com/scholar?q=Norma+Faust%2E+1973%2E+Lecciones+para+el+aprendizaje+del+idioma+shipibo%2Dconibo%2C+volume+1+of+Documento+de+Trabajo%2E+Instituto+Ling%C3%BC%C3%ADstico+de+Verano%2C+Yarinacocha%2E)
25. Isaac Feldman and Rolando Coto-Solano. 2020. Neural machine translation models with back-translation for the extremely low-resource indigenous language Bribri. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3965–3976, Barcelona, Spain (Online). International… [find](https://scholar.google.com/scholar?q=Isaac+Feldman+and+Rolando+Coto%2DSolano%2E+2020%2E+Neural+machine+translation+models+with+back%2Dtranslation+for+the+extremely+low%2Dresource+indigenous+language+Bribri%2E+In+Proceedings+of+the+28th+International)
26. Sofía Flores Solórzano. 2017. Corpus oral pandialectal de la lengua bribri. [http://bribri.net](http://bribri.net). [↗](http://bribri.net)
27. Ana-Paula Galarreta, Andrés Melgar, and Arturo Oncevay. 2017. Corpus creation and initial SMT experiments between Spanish and Shipibo-konibo. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 238–244, Varna, Bulgaria. INCOMA Ltd. [find](https://scholar.google.com/scholar?q=Ana%2DPaula+Galarreta%2C+Andr%C3%A9s+Melgar%2C+and+Arturo+Oncevay%2E+2017%2E+Corpus+creation+and+initial+SMT+experiments+between+Spanish+and+Shipibo%2Dkonibo%2E+In+Proceedings+of+the+International+Conference+Recent+Adva)
28. Ives Goddard. 1996. Introduccion. In William C. Sturtevant, editor, Handbook of North American Indians (vol. 17), chapter 1, pages 1–6. University of Texas. [find](https://scholar.google.com/scholar?q=Ives+Goddard%2E+1996%2E+Introduccion%2E+In+William+C%2E+Sturtevant%2C+editor%2C+Handbook+of+North+American+Indians+%28vol%2E+17%29%2C+chapter+1%2C+pages+1%E2%80%936%2E+University+of+Texas%2E)
29. Héctor Erasmo Gómez Montoya, Kervy Dante Rivas Rojas, and Arturo Oncevay. 2019. A continuous improvement framework of machine translation for Shipibo-konibo. In Proceedings of the 2nd Workshop on Technologies for MT of Low Resource Languages, pages 17–23, Dublin, Ireland. European Association for Ma… [find](https://scholar.google.com/scholar?q=H%C3%A9ctor+Erasmo+G%C3%B3mez+Montoya%2C+Kervy+Dante+Rivas+Rojas%2C+and+Arturo+Oncevay%2E+2019%2E+A+continuous+improvement+framework+of+machine+translation+for+Shipibo%2Dkonibo%2E+In+Proceedings+of+the+2nd+Workshop+on+Tech)
30. Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning word vectors for 157 languages. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018). [find](https://scholar.google.com/scholar?q=Edouard+Grave%2C+Piotr+Bojanowski%2C+Prakhar+Gupta%2C+Armand+Joulin%2C+and+Tomas+Mikolov%2E+2018%2E+Learning+word+vectors+for+157+languages%2E+In+Proceedings+of+the+International+Conference+on+Language+Resources+an) [our card](https://md.research.elamur.ai/free/en/papers/grave-2018-learning-word-vectors-157-languages/)
31. Joseph Harold Greenberg. 1963. guage. [find](https://scholar.google.com/scholar?q=Joseph+Harold+Greenberg%2E+1963%2E+guage%2E)
32. Universals of lan-Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Hum… [find](https://scholar.google.com/scholar?q=Universals+of+lan%2DSuchin+Gururangan%2C+Swabha+Swayamdipta%2C+Omer+Levy%2C+Roy+Schwartz%2C+Samuel+Bowman%2C+and+Noah+A%2E+Smith%2E+2018%2E+Annotation+artifacts+in+natural+language+inference+data%2E+In+Proceedings+of+the)
33. Ximena Gutierrez-Vasques, Gerardo Sierra, and Isaac Hernandez Pompa. 2016. Axolotl: a web accessible parallel corpus for Spanish-Nahuatl. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4210–4214, Portorož, Slovenia. European Language Resour… [find](https://scholar.google.com/scholar?q=Ximena+Gutierrez%2DVasques%2C+Gerardo+Sierra%2C+and+Isaac+Hernandez+Pompa%2E+2016%2E+Axolotl%3A+a+web+accessible+parallel+corpus+for+Spanish%2DNahuatl%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Langua)
34. Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato. 2019. The flores evaluation datasets for low-resource machine translation: Nepali–english and sinhala–english. In Proceedings of the 2019 Conference on Empirical Method… [find](https://scholar.google.com/scholar?q=Francisco+Guzm%C3%A1n%2C+Peng%2DJen+Chen%2C+Myle+Ott%2C+Juan+Pino%2C+Guillaume+Lample%2C+Philipp+Koehn%2C+Vishrav+Chaudhary%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2019%2E+The+flores+evaluation+datasets+for+low%2Dresource+machine+transl) [our card](https://md.research.elamur.ai/free/en/papers/guzman-2019-flores-evaluation-datasets-low-resource/)
35. Petr Homola. 2012. Building a formal grammar for a polysynthetic language. In Formal Grammar, pages 228–242, Berlin, Heidelberg. Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Petr+Homola%2E+2012%2E+Building+a+formal+grammar+for+a+polysynthetic+language%2E+In+Formal+Grammar%2C+pages+228%E2%80%93242%2C+Berlin%2C+Heidelberg%2E+Springer+Berlin+Heidelberg%2E)
36. Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalization. CoRR, abs/2003.11080. [find](https://scholar.google.com/scholar?q=Junjie+Hu%2C+Sebastian+Ruder%2C+Aditya+Siddhant%2C+Graham+Neubig%2C+Orhan+Firat%2C+and+Melvin+Johnson%2E+2020%2E+Xtreme%3A+A+massively+multilingual+multi%2Dtask+benchmark+for+evaluating+cross%2Dlingual+generalization%2E+Co) [our card](https://md.research.elamur.ai/free/en/papers/hu-2020-xtreme-massively-multilingual-multi-task/)
37. INALI. 2009. Catálogo de las lenguas indígenas nacionales: Variantes lingüísticas de méxico con sus autodenominaciones y referencias geoestadísticas. INALI. [find](https://scholar.google.com/scholar?q=INALI%2E+2009%2E+Cat%C3%A1logo+de+las+lenguas+ind%C3%ADgenas+nacionales%3A+Variantes+ling%C3%BC%C3%ADsticas+de+m%C3%A9xico+con+sus+autodenominaciones+y+referencias+geoestad%C3%ADsticas%2E+INALI%2E)
38. INALI. 2014. Norma de escritura de la Lengua Hñähñu (Otomí), 1st edition. Secretaria de culatura. [find](https://scholar.google.com/scholar?q=INALI%2E+2014%2E+Norma+de+escritura+de+la+Lengua+H%C3%B1%C3%A4h%C3%B1u+%28Otom%C3%AD%29%2C+1st+edition%2E+Secretaria+de+culatura%2E)
39. INALI. 2017. Etnografía del pueblo tarahumara (rarámuri). [find](https://scholar.google.com/scholar?q=INALI%2E+2017%2E+Etnograf%C3%ADa+del+pueblo+tarahumara+%28rar%C3%A1muri%29%2E)
40. INEC. 2011. Población total en territorios indígenas por autoidentificación a la etnia indígena y habla de alguna lengua indígena, según pueblo y territorio indígena. In Instituto Nacional de Estadística y Censos, editor, Censo 2011. [find](https://scholar.google.com/scholar?q=INEC%2E+2011%2E+Poblaci%C3%B3n+total+en+territorios+ind%C3%ADgenas+por+autoidentificaci%C3%B3n+a+la+etnia+ind%C3%ADgena+y+habla+de+alguna+lengua+ind%C3%ADgena%2C+seg%C3%BAn+pueblo+y+territorio+ind%C3%ADgena%2E+In+Instituto+Nacional+de+Estad%C3%ADst)
41. INEGI. 2008. Catálogo de las lenguas indígenas nacionales: Variantes lingüísticas de méxico con sus autodenominaciones y referencias geoestadísticas. Diario Oficial, pages 31–108. [find](https://scholar.google.com/scholar?q=INEGI%2E+2008%2E+Cat%C3%A1logo+de+las+lenguas+ind%C3%ADgenas+nacionales%3A+Variantes+ling%C3%BC%C3%ADsticas+de+m%C3%A9xico+con+sus+autodenominaciones+y+referencias+geoestad%C3%ADsticas%2E+Diario+Oficial%2C+pages+31%E2%80%93108%2E)
42. José L. Iturrioz and Paula Gómez-López. 2008. Gramatica wixarika i. [find](https://scholar.google.com/scholar?q=Jos%C3%A9+L%2E+Iturrioz+and+Paula+G%C3%B3mez%2DL%C3%B3pez%2E+2008%2E+Gramatica+wixarika+i%2E)
43. Carla Victoria Jara Murillo. 2018a. Gramática de la Lengua Bribri. EDigital. [find](https://scholar.google.com/scholar?q=Carla+Victoria+Jara+Murillo%2E+2018a%2E+Gram%C3%A1tica+de+la+Lengua+Bribri%2E+EDigital%2E)
44. Carla Victoria Jara Murillo. 2018b. I Ttè Historias Bribris, second edition. Editorial de la Universidad de Costa Rica. [find](https://scholar.google.com/scholar?q=Carla+Victoria+Jara+Murillo%2E+2018b%2E+I+Tt%C3%A8+Historias+Bribris%2C+second+edition%2E+Editorial+de+la+Universidad+de+Costa+Rica%2E)
45. Carla Victoria Jara Murillo and Alí García Segura. 2013. Se’ ttö’ bribri ie Hablemos en bribri. EDigital. [find](https://scholar.google.com/scholar?q=Carla+Victoria+Jara+Murillo+and+Al%C3%AD+Garc%C3%ADa+Segura%2E+2013%2E+Se%E2%80%99+tt%C3%B6%E2%80%99+bribri+ie+Hablemos+en+bribri%2E+EDigital%2E)
46. Katharina Kann, Kyunghyun Cho, and Samuel R. Bowman. 2019. Towards realistic practices in low-resource natural language processing: The development set. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan… [find](https://scholar.google.com/scholar?q=Katharina+Kann%2C+Kyunghyun+Cho%2C+and+Samuel+R%2E+Bowman%2E+2019%2E+Towards+realistic+practices+in+low%2Dresource+natural+language+processing%3A+The+development+set%2E+In+Proceedings+of+the+2019+Conference+on+Empiri)
47. Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71, Brussels, Belgium. Associ… [find](https://scholar.google.com/scholar?q=Taku+Kudo+and+John+Richardson%2E+2018%2E+SentencePiece%3A+A+simple+and+language+independent+subword+tokenizer+and+detokenizer+for+neural+text+processing%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+M) [our card](https://md.research.elamur.ai/free/en/papers/kudo-2018-sentencepiece-simple-language-independent-subword/)
48. Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700. [find](https://scholar.google.com/scholar?q=Alexandre+Lacoste%2C+Alexandra+Luccioni%2C+Victor+Schmidt%2C+and+Thomas+Dandres%2E+2019%2E+Quantifying+the+carbon+emissions+of+machine+learning%2E+arXiv+preprint+arXiv%3A1910%2E09700%2E) [our card](https://md.research.elamur.ai/free/en/papers/lacoste-2019-quantifying-carbon-emissions-machine-learning/)
49. Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. In NeurIPS. [find](https://scholar.google.com/scholar?q=Guillaume+Lample+and+Alexis+Conneau%2E+2019%2E+Cross%2Dlingual+language+model+pretraining%2E+In+NeurIPS%2E)
50. Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2018a. Unsupervised machine translation using monolingual corpora only. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Alexis+Conneau%2C+Ludovic+Denoyer%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2018a%2E+Unsupervised+machine+translation+using+monolingual+corpora+only%2E+In+International+Conference+on+Learning+Representat) [our card](https://md.research.elamur.ai/free/en/papers/lample-2017-unsupervised-machine-translation-using-monolingual/)
51. Guillaume Lample, Alexis Conneau, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018b. Word translation without parallel data. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net… [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Alexis+Conneau%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Ludovic+Denoyer%2C+and+Herv%C3%A9+J%C3%A9gou%2E+2018b%2E+Word+translation+without+parallel+data%2E+In+6th+International+Conference+on+Learning+Representations%2C+IC)
52. Anne Lauscher, Vinit Ravishankar, Ivan Vulić, and Goran Glavaš. 2020. From zero to hero: On the limitations of zero-shot language transfer with multilingual Transformers. pages 4483–4499. [find](https://scholar.google.com/scholar?q=Anne+Lauscher%2C+Vinit+Ravishankar%2C+Ivan+Vulic%CC%81%2C+and+Goran+Glava%C5%A1%2E+2020%2E+From+zero+to+hero%3A+On+the+limitations+of+zero%2Dshot+language+transfer+with+multilingual+Transformers%2E+pages+4483%E2%80%934499%2E) [our card](https://md.research.elamur.ai/free/en/papers/lauscher-2020-zero-hero-limitations-zero-shot/)
53. Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, and Mi… [find](https://scholar.google.com/scholar?q=Yaobo+Liang%2C+Nan+Duan%2C+Yeyun+Gong%2C+Ning+Wu%2C+Fenfei+Guo%2C+Weizhen+Qi%2C+Ming+Gong%2C+Linjun+Shou%2C+Daxin+Jiang%2C+Guihong+Cao%2C+Xiaodong+Fan%2C+Ruofei+Zhang%2C+Rahul+Agrawal%2C+Edward+Cui%2C+Sining+Wei%2C+Taroon+Bharti%2C+)
54. Y. Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692. [find](https://scholar.google.com/scholar?q=Y%2E+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+M%2E+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+ArXiv%2C+)
55. James Loriot, Erwin Lauriout, and Dwight Day. 1993. Diccionario Shipibo-Castellano. Instituto Lingüístico de Verano. [find](https://scholar.google.com/scholar?q=James+Loriot%2C+Erwin+Lauriout%2C+and+Dwight+Day%2E+1993%2E+Diccionario+Shipibo%2DCastellano%2E+Instituto+Ling%C3%BC%C3%ADstico+de+Verano%2E)
56. Carl Lumholtz. 2011. Unknown Mexico: A Record of Five Years’ Exploration Among the Tribes of the Western Sierra Madre, volume 2. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Carl+Lumholtz%2E+2011%2E+Unknown+Mexico%3A+A+Record+of+Five+Years%E2%80%99+Exploration+Among+the+Tribes+of+the+Western+Sierra+Madre%2C+volume+2%2E+Cambridge+University+Press%2E)
57. Jeff MacSwan. 1998. The argument status of nps in southeast puebla nahuatl: Comments on the polysynthesis parameter. Southwest Journal of Linguistics, 17(2):101–114. [find](https://scholar.google.com/scholar?q=Jeff+MacSwan%2E+1998%2E+The+argument+status+of+nps+in+southeast+puebla+nahuatl%3A+Comments+on+the+polysynthesis+parameter%2E+Southwest+Journal+of+Linguistics%2C+17%282%29%3A101%E2%80%93114%2E)
58. Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Processing+%28EMNLP%29%2C+pages+1532%E2%80%931543%2C+Doha%2C+Qatar%2E+Association+for+Computational+Linguistics%2E)
59. Manuel Mager, Dionico Gonzalez, and Ivan Meza. 2017. Probabilistic finite-state morphological segmenter for wixarika (huichol). [find](https://scholar.google.com/scholar?q=Manuel+Mager%2C+Dionico+Gonzalez%2C+and+Ivan+Meza%2E+2017%2E+Probabilistic+finite%2Dstate+morphological+segmenter+for+wixarika+%28huichol%29%2E)
60. Manuel Mager, Ximena Gutierrez-Vasques, Gerardo Sierra, and Ivan Meza-Ruiz. 2018. Challenges of language technologies for the indigenous languages of the Americas. In Proceedings of the 27th International Conference on Computational Linguistics, pages 55–69, Santa Fe, New Mexico, USA. Association fo… [find](https://scholar.google.com/scholar?q=Manuel+Mager%2C+Ximena+Gutierrez%2DVasques%2C+Gerardo+Sierra%2C+and+Ivan+Meza%2DRuiz%2E+2018%2E+Challenges+of+language+technologies+for+the+indigenous+languages+of+the+Americas%2E+In+Proceedings+of+the+27th+Internati)
61. Jesus Manuel Mager-Hois. 2017. Traductor hıbrido wixárika-espanol con escasos recursos bilingües. Ph.D. thesis, Master’s thesis, Universidad Autónoma Metropolitana. [find](https://scholar.google.com/scholar?q=Jesus+Manuel+Mager%2DHois%2E+2017%2E+Traductor+h%C4%B1brido+wix%C3%A1rika%2Despanol+con+escasos+recursos+biling%C3%BCes%2E+Ph%2ED%2E+thesis%2C+Master%E2%80%99s+thesis%2C+Universidad+Aut%C3%B3noma+Metropolitana%2E)
62. Enrique Margery. 2005. Diccionario Fraseológico Bribri-Español Español-Bribri, second edition. Editorial de la Universidad de Costa Rica. [find](https://scholar.google.com/scholar?q=Enrique+Margery%2E+2005%2E+Diccionario+Fraseol%C3%B3gico+Bribri%2DEspa%C3%B1ol+Espa%C3%B1ol%2DBribri%2C+second+edition%2E+Editorial+de+la+Universidad+de+Costa+Rica%2E)
63. Bartomeu Melià. 1992. La lengua Guaraní del Paraguay: Historia, sociedad y literatura. Editorial MAPFRE, Madrid. [find](https://scholar.google.com/scholar?q=Bartomeu+Meli%C3%A0%2E+1992%2E+La+lengua+Guaran%C3%AD+del+Paraguay%3A+Historia%2C+sociedad+y+literatura%2E+Editorial+MAPFRE%2C+Madrid%2E)
64. Elena Mihas. 2017. The kampa subgroup of the arawak language family. The Cambridge Handbook of Linguistic Typology, page 782–814. [find](https://scholar.google.com/scholar?q=Elena+Mihas%2E+2017%2E+The+kampa+subgroup+of+the+arawak+language+family%2E+The+Cambridge+Handbook+of+Linguistic+Typology%2C+page+782%E2%80%93814%2E)
65. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, volume 26, pages 3111–3119. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Advances+in+Neural+Information+Processing+)
66. Christopher Moseley. 2010. Atlas of the World’s Languages in Danger. Unesco. [find](https://scholar.google.com/scholar?q=Christopher+Moseley%2E+2010%2E+Atlas+of+the+World%E2%80%99s+Languages+in+Danger%2E+Unesco%2E)
67. B. Muller, Antonis Anastasopoulos, Benoît Sagot, and Djamé Seddah. 2020. When being unseen from mbert is just the beginning: Handling new languages with multilingual language models. ArXiv, abs/2010.12858. [find](https://scholar.google.com/scholar?q=B%2E+Muller%2C+Antonis+Anastasopoulos%2C+Beno%C3%AEt+Sagot%2C+and+Djam%C3%A9+Seddah%2E+2020%2E+When+being+unseen+from+mbert+is+just+the+beginning%3A+Handling+new+languages+with+multilingual+language+models%2E+ArXiv%2C+abs%2F2010%2E1) [our card](https://md.research.elamur.ai/free/en/papers/muller-2021-when-being-unseen-mbert-is/)
68. Johanna Nichols. 1986. Head-marking and dependent-marking grammar. Language, 62(1):56–119. [find](https://scholar.google.com/scholar?q=Johanna+Nichols%2E+1986%2E+Head%2Dmarking+and+dependent%2Dmarking+grammar%2E+Language%2C+62%281%29%3A56%E2%80%93119%2E)
69. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations. [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Sergey+Edunov%2C+Alexei+Baevski%2C+Angela+Fan%2C+Sam+Gross%2C+Nathan+Ng%2C+David+Grangier%2C+and+Michael+Auli%2E+2019%2E+fairseq%3A+A+fast%2C+extensible+toolkit+for+sequence+modeling%2E+In+Proceedings+of+NAACL%2DHL) [our card](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
70. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
71. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020a. MAD-X: An Adapter-Based Framework… [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+Manning%2E+2014%2E+GloVe%3A+Global+vectors+for+word+representation%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Jonas+P)
72. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020b. Unks everywhere: Adapting multilingual language models to new scripts. [find](https://scholar.google.com/scholar?q=Jonas+Pfeiffer%2C+Ivan+Vulic%CC%81%2C+Iryna+Gurevych%2C+and+Sebastian+Ruder%2E+2020b%2E+Unks+everywhere%3A+Adapting+multilingual+language+models+to+new+scripts%2E) [our card](https://md.research.elamur.ai/free/en/papers/pfeiffer-2021-unks-everywhere-adapting-multilingual-language/)
73. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996– 5001, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Telmo+Pires%2C+Eva+Schlinger%2C+and+Dan+Garrette%2E+2019%2E+How+multilingual+is+multilingual+BERT%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+4996%E2%80%93+5001%2C) [our card](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/)
74. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Maja+Popovic%CC%81%2E+2015%2E+chrF%3A+character+n%2Dgram+F%2Dscore+for+automatic+MT+evaluation%2E+In+Proceedings+of+the+Tenth+Workshop+on+Statistical+Machine+Translation%2C+pages+392%E2%80%93395%2C+Lisbon%2C+Portugal%2E+Association+f) [our card](https://md.research.elamur.ai/free/en/papers/popovic-2015-chrf-character-n-gram-f/)
75. Carlos Sánchez Avendaño. 2013. Lenguas en peligro en Costa Rica: vitalidad, documentación y descripción. Revista Káñina, 37(1):219–250. [find](https://scholar.google.com/scholar?q=Carlos+S%C3%A1nchez+Avenda%C3%B1o%2E+2013%2E+Lenguas+en+peligro+en+Costa+Rica%3A+vitalidad%2C+documentaci%C3%B3n+y+descripci%C3%B3n%2E+Revista+K%C3%A1%C3%B1ina%2C+37%281%29%3A219%E2%80%93250%2E)
76. SEGOB. 2020a. Sistema de Información Cultural -Lenguas indígenas: Huichol . [https://sic.gob.mx/ficha.php?](https://sic.gob.mx/ficha.php?) table=inali_li. [↗](https://sic.gob.mx/ficha.php?)
77. SEGOB. 2020b. Sistema de Información Cultural - Lenguas indígenas: Nnahuatl. [https://sic.gob.mx/ficha.php?](https://sic.gob.mx/ficha.php?) table=inali_li&table_id=5. [↗](https://sic.gob.mx/ficha.php?)
78. SEGOB. 2020c. Sistema de Información Cultural - Lenguas indígenas: Tarahumara. [http://sic.gob.mx/ficha.php?table=](http://sic.gob.mx/ficha.php?table=) inali_li&table_id=15. [↗](http://sic.gob.mx/ficha.php?table=)
79. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Ling)
80. Rémi Siméon. 1977. Diccionario de la lengua náhuatl o mexicana, volume 1. Siglo XXI. [find](https://scholar.google.com/scholar?q=R%C3%A9mi+Sim%C3%A9on%2E+1977%2E+Diccionario+de+la+lengua+n%C3%A1huatl+o+mexicana%2C+volume+1%2E+Siglo+XXI%2E)
81. Thelma D Sullivan and Miguel León-Portilla. 1976. Compendio de la gramática náhuatl, volume 18. Universidad nacional autónoma de México, Instituto de investigaciones históricas. [find](https://scholar.google.com/scholar?q=Thelma+D+Sullivan+and+Miguel+Le%C3%B3n%2DPortilla%2E+1976%2E+Compendio+de+la+gram%C3%A1tica+n%C3%A1huatl%2C+volume+18%2E+Universidad+nacional+aut%C3%B3noma+de+M%C3%A9xico%2C+Instituto+de+investigaciones+hist%C3%B3ricas%2E)
82. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In Proceedings of the Eighth International Conference on Language Resources and System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=J%C3%B6rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+OPUS%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Language+Resources+and+System+Demonstrations%2C+pages+38%E2%80%9345%2C+Online%2E+Associati)
83. Evaluation (LREC’12), pages 2214–2218, Istanbul, Turkey. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Evaluation+%28LREC%E2%80%9912%29%2C+pages+2214%E2%80%932218%2C+Istanbul%2C+Turkey%2E+European+Language+Resources+Association+%28ELRA%29%2E)
84. Pilar Valenzuela. 2003. Transitivity in Shipibo-Konibo grammar. Ph.D. thesis, University of Oregon. [find](https://scholar.google.com/scholar?q=Pilar+Valenzuela%2E+2003%2E+Transitivity+in+Shipibo%2DKonibo+grammar%2E+Ph%2ED%2E+thesis%2C+University+of+Oregon%2E)
85. Alonso Vasquez, Renzo Ego Aguirre, Candy Angulo, John Miller, Claudia Villanueva, Željko Agić, Roberto Zariquiey, and Arturo Oncevay. 2018. Toward Universal Dependencies for Shipibo-konibo. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 151–161, Brussels, Belgium.… [find](https://scholar.google.com/scholar?q=Alonso+Vasquez%2C+Renzo+Ego+Aguirre%2C+Candy+Angulo%2C+John+Miller%2C+Claudia+Villanueva%2C+%C5%BDeljko+Agic%CC%81%2C+Roberto+Zariquiey%2C+and+Arturo+Oncevay%2E+2018%2E+Toward+Universal+Dependencies+for+Shipibo%2Dkonibo%2E+In+Procee)
86. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+Neural+Information+Process)
87. Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 833–844,… [find](https://scholar.google.com/scholar?q=Shijie+Wu+and+Mark+Dredze%2E+2019%2E+Beto%2C+bentz%2C+becas%3A+The+surprising+cross%2Dlingual+effectiveness+of+BERT%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+t) [our card](https://md.research.elamur.ai/free/en/papers/wu-2019-beto-bentz-becas-surprising-cross/)
88. Shijie Wu and Mark Dredze. 2020. Are all languages created equal in multilingual BERT? In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 120–130, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shijie+Wu+and+Mark+Dredze%2E+2020%2E+Are+all+languages+created+equal+in+multilingual+BERT%3F+In+Proceedings+of+the+5th+Workshop+on+Representation+Learning+for+NLP%2C+pages+120%E2%80%93130%2C+Online%2E+Association+for+Com) [our card](https://md.research.elamur.ai/free/en/papers/wu-2020-are-all-languages-created-equal/)
89. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Alex+Wang%2C+Yada+Pruksachatkun%2C+Nikita+Nangia%2C+Amanpreet+Singh%2C+Julian+Michael%2C+Felix+Hill%2C+Omer+Levy%2C+and+Samuel+Bowman%2E+2019%2E+Superglue%3A+A+stickier+benchmark+for+general%2Dpurpose+language+understandin) [our card](https://md.research.elamur.ai/free/en/papers/wang-2019-superglue-stickier-benchmark-general-purpose/)
90. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355,… [find](https://scholar.google.com/scholar?q=Alex+Wang%2C+Amanpreet+Singh%2C+Julian+Michael%2C+Felix+Hill%2C+Omer+Levy%2C+and+Samuel+Bowman%2E+2018%2E+GLUE%3A+A+multi%2Dtask+benchmark+and+analysis+platform+for+natural+language+understanding%2E+In+Proceedings+of+the) [our card](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/)
91. Zihan Wang, Karthikeyan K, Stephen Mayhew, and Dan Roth. 2020. Extending multilingual BERT to low-resource languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2649–2656, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zihan+Wang%2C+Karthikeyan+K%2C+Stephen+Mayhew%2C+and+Dan+Roth%2E+2020%2E+Extending+multilingual+BERT+to+low%2Dresource+languages%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2C+pages+26)
92. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa… [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chap)
93. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho… [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+R%C3%A9mi+Louf%2C+Morgan+Funtowicz%2C+Joe+Davison%2C+Sam+Shleifer%2C+Patrick+von+Platen%2C+Clara+) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)