---
title: "Parsing with Multilingual BERT, a Small Corpus, and a Small"
description: "Varieties that are barely present in, or absent from, the pretraining data have too little labeled and… 42 references, cited by 60 papers here."
authors: ["Ethan C. Chau", "Lucy H. Lin", "Noah A. Smith"]
year: 2020
venue: "Findings of the Association for Computational Linguistics: EMNLP 2020"
source: "https://aclanthology.org/2020.findings-emnlp.118/"
doi: "10.18653/v1/2020.findings-emnlp.118"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chau-2020-parsing-multilingual-bert-small-corpus/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chau-2020-parsing-multilingual-bert-small-corpus/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) / Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank

# Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank

**Authors:** Ethan C. Chau, Lucy H. Lin and [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) · **Year:** 2020 · **Venue:** [Findings of the Association for Computational Linguistics: EMNLP 2020](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2020](https://md.research.elamur.ai/free/en/venues/findings/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.findings-emnlp.118/) · [PDF](https://aclanthology.org/2020.findings-emnlp.118.pdf) · [DOI 10.18653/v1/2020.findings-emnlp.118](https://doi.org/10.18653/v1/2020.findings-emnlp.118) · [OpenAlex](https://openalex.org/W3089959042)

**Citations:** 43 citations (OpenAlex, 2026-10-14) · [60](https://md.research.elamur.ai/free/en/papers/chau-2020-parsing-multilingual-bert-small-corpus/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Multilingual models such as mBERT serve low-resource language varieties unevenly. Varieties that are barely present in, or absent from, the pretraining data have too little labeled and unlabeled text to train a strong monolingual model. Chau, Lin, and Smith test whether mBERT can be adapted cheaply using only a small unlabeled corpus in the target variety. They propose three methods:

- **Language-adaptive pretraining (LAPT):** extra masked-LM training on target-language text.
- **Vocabulary augmentation (VA):** fills mBERT's unused vocabulary slots with frequent target-language wordpieces before LAPT.
- **Tiered VA (TVA):** the same as VA, but with a higher learning rate for the new embeddings.

They evaluate the methods on UD dependency parsing in Irish, Maltese, Vietnamese and Singlish, using a biaffine parser on top of mBERT layer representations, with frozen and finetuned variants. LAPT reliably beats unadapted mBERT and fastText/ELMo baselines. VA adds smaller but significant gains, while TVA rarely helps beyond VA. The gains are largest for varieties poorly covered by mBERT's pretraining data. This shows that a little unlabeled text can substantially narrow the gap multilingual models show on low-resource languages.

## Key points
- The authors group target varieties by their relation to mBERT's pretraining data:
  - **Type 0:** Vietnamese, plus Singlish, whose component languages are well represented.
  - **Type 1:** Irish, which has a tiny Wikipedia.
  - **Type 2:** Maltese, which mBERT never saw.
- VA repurposes mBERT's 99 "unused" vocabulary tokens for new wordpieces, so the pretrained architecture is left unchanged. Unlabeled data comes from Wikipedia, with only a 5% sample for Vietnamese, and from the SG Talk Forum for Singlish.
- For Maltese, LAS rises from 67.06 (frozen mBERT) to 78.51 with LAPT. With finetuning it rises from 76.74 (mBERT+FT) to 83.53 (VA+FT). For Irish, finetuned LAS rises from 72.67 to 76.17 with VA+FT.
- Maximum relative error reduction over unadapted mBERT tracks language type: about 39% for Maltese, 15% for Irish, 11% for Singlish and 5% for Vietnamese.
- Code, data splits and hyperparameters are released at github.com/ethch18/parsing-mbert.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 263 works · 180,581 citations
- [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) — Research institute, Seattle, United States · 219 works · 89,712 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,344 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 465 works · [🗂️ Treebanks](https://md.research.elamur.ai/free/en/topics/treebanks/) · 118 works

## Cite

Cite as: Ethan C. Chau, Lucy H. Lin and Noah A. Smith. 2020. Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank. Findings of the Association for Computational Linguistics: EMNLP 2020.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{chau2020parsing,
  title = {{Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank}},
  author = {Ethan C. Chau, Lucy H. Lin and Noah A. Smith},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.118},
  url = {https://doi.org/10.18653/v1/2020.findings-emnlp.118}
}
```

- [← previous An Annotated Dataset of Coreference in English Literature](https://md.research.elamur.ai/free/en/papers/bamman-2020-annotated-dataset-coreference-english-literature/)
- [next → Uncertain Natural Language Inference](https://md.research.elamur.ai/free/en/papers/chen-2020-uncertain-natural-language-inference/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/)

## References

42 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rodrigo Agerri, Iñaki San Vicente, Jon Ander Campos, Ander Barrena, Xabier Saralegi, Aitor Soroa, and Eneko Agirre. 2020. Give your text representation models some love: the case for Basque. In Proc. of LREC. [find](https://scholar.google.com/scholar?q=Rodrigo+Agerri%2C+In%CC%83aki+San+Vicente%2C+Jon+Ander+Campos%2C+Ander+Barrena%2C+Xabier+Saralegi%2C+Aitor+Soroa%2C+and+Eneko+Agirre%2E+2020%2E+Give+your+text+representation+models+some+love%3A+the+case+for+Basque%2E+In+Proc%2E)
2. Emily Alsentzer, John Murphy, William Boag, Wei-Hung Weng, Di Jindi, Tristan Naumann, and Matthew McDermott. 2019. [Publicly available clinical BERT embeddings](https://md.research.elamur.ai/free/en/papers/alsentzer-2019-publicly-available-clinical-bert-embeddings/). In Proc. of Clinical Natural Language Processing Workshop.
3. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. [On the cross-lingual transferability of monolingual representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/). In Proc. of ACL.
4. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: Pretrained language model for scientific text. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Iz+Beltagy%2C+Kyle+Lo%2C+and+Arman+Cohan%2E+2019%2E+SciBERT%3A+Pretrained+language+model+for+scientific+text%2E+In+Proc%2E+of+EMNLP%2E)
5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). TACL, 5:135–146.
6. Steven Cao, Nikita Kitaev, and Dan Klein. 2020. Multilingual alignment of contextual word representations. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Steven+Cao%2C+Nikita+Kitaev%2C+and+Dan+Klein%2E+2020%2E+Multilingual+alignment+of+contextual+word+representations%2E+In+Proc%2E+of+ICLR%2E)
7. Tuhin Chakrabarty, Christopher Hidey, and Kathy McKeown. 2019. IMHO fine-tuning improves claim detection. In Proc. of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Tuhin+Chakrabarty%2C+Christopher+Hidey%2C+and+Kathy+McKeown%2E+2019%2E+IMHO+fine%2Dtuning+improves+claim+detection%2E+In+Proc%2E+of+NAACL%2DHLT%2E)
8. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020a. Unsupervised cross-lingual representation learning at scale. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzma%CC%81n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020a%2E+Unsupervised+cross%2Dlin)
9. Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. 2020b. [Emerging cross-lingual structure in pretrained language models](https://md.research.elamur.ai/free/en/papers/conneau-2020-emerging-cross-lingual-structure-pretrained/). In Proc. of ACL.
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proc. of NAACL-HLT.
11. Jesse Dodge, Suchin Gururangan, Dallas Card, Roy Schwartz, and Noah A. Smith. 2019. Show your work: Improved reporting of experimental results. In Proc. of EMNLP-IJCNLP. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Suchin+Gururangan%2C+Dallas+Card%2C+Roy+Schwartz%2C+and+Noah+A%2E+Smith%2E+2019%2E+Show+your+work%3A+Improved+reporting+of+experimental+results%2E+In+Proc%2E+of+EMNLP%2DIJCNLP%2E)
12. Timothy Dozat and Christopher D. Manning. 2017. [Deep biaffine attention for neural dependency parsing](https://md.research.elamur.ai/free/en/papers/dozat-2016-deep-biaffine-attention-neural-dependency/). In Proc. of ICLR.
13. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. [AllenNLP: A deep semantic natural language processing platform](https://md.research.elamur.ai/free/en/papers/gardner-2018-allennlp-deep-semantic-natural-language/). In Proc. of Workshop for NLP Open Source Software (NLP-OSS).
14. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Ana+Marasovic%CC%81%2C+Swabha+Swayamdipta%2C+Kyle+Lo%2C+Iz+Beltagy%2C+Doug+Downey%2C+and+Noah+A%2E+Smith%2E+2020%2E+Don%E2%80%99t+stop+pretraining%3A+Adapt+language+models+to+domains+and+tasks%2E+In+Proc%2E+of+ACL%2E)
15. Xiaochuang Han and Jacob Eisenstein. 2019. [Unsupervised domain adaptation of contextualized embeddings for sequence labeling](https://md.research.elamur.ai/free/en/papers/han-2019-unsupervised-domain-adaptation-contextualized-embeddings/). In Proc. of EMNLP.
16. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Jeremy+Howard+and+Sebastian+Ruder%2E+2018%2E+Universal+language+model+fine%2Dtuning+for+text+classification%2E+In+Proc%2E+of+ACL%2E)
17. Karthikeyan K, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020. Cross-lingual ability of multilingual BERT: An empirical study. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Karthikeyan+K%2C+Zihan+Wang%2C+Stephen+Mayhew%2C+and+Dan+Roth%2E+2020%2E+Cross%2Dlingual+ability+of+multilingual+BERT%3A+An+empirical+study%2E+In+Proc%2E+of+ICLR%2E)
18. Dan Kondratyuk and Milan Straka. 2019. [75 languages, 1 model: Parsing Universal Dependencies universally](https://md.research.elamur.ai/free/en/papers/kondratyuk-2019-75-languages-1-model-parsing/). In Proc. of EMNLP-IJCNLP.
19. Surafel M. Lakew, Aliia Erofeeva, Matteo Negri, Marcello Federico, and Marco Turchi. 2018. Transfer learning in multilingual neural machine translation with dynamic vocabulary. In Proc. of IWSLT. [find](https://scholar.google.com/scholar?q=Surafel+M%2E+Lakew%2C+Aliia+Erofeeva%2C+Matteo+Negri%2C+Marcello+Federico%2C+and+Marco+Turchi%2E+2018%2E+Transfer+learning+in+multilingual+neural+machine+translation+with+dynamic+vocabulary%2E+In+Proc%2E+of+IWSLT%2E)
20. Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. In Proc. of NeurIPS. [find](https://scholar.google.com/scholar?q=Guillaume+Lample+and+Alexis+Conneau%2E+2019%2E+Cross%2Dlingual+language+model+pretraining%2E+In+Proc%2E+of+NeurIPS%2E)
21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692 [cs.CL]. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+RoBERTa%3A+A+robustly+optimized+BERT+pretraining+approach%2E+a)
22. Lajanugen Logeswaran, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, Jacob Devlin, and Honglak Lee. 2019. Zero-shot entity linking by reading entity descriptions. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Lajanugen+Logeswaran%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+Kristina+Toutanova%2C+Jacob+Devlin%2C+and+Honglak+Lee%2E+2019%2E+Zero%2Dshot+entity+linking+by+reading+entity+descriptions%2E+In+Proc%2E+of+ACL%2E)
23. David Mueller, Nicholas Andrews, and Mark Dredze. 2020. Sources of transfer in multilingual named entity recognition. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=David+Mueller%2C+Nicholas+Andrews%2C+and+Mark+Dredze%2E+2020%2E+Sources+of+transfer+in+multilingual+named+entity+recognition%2E+In+Proc%2E+of+ACL%2E)
24. Phoebe Mulcaire, Jungo Kasai, and Noah A. Smith. 2019a. Low-resource parsing with crosslingual contextualized representations. In Proc. of CoNLL. [find](https://scholar.google.com/scholar?q=Phoebe+Mulcaire%2C+Jungo+Kasai%2C+and+Noah+A%2E+Smith%2E+2019a%2E+Low%2Dresource+parsing+with+crosslingual+contextualized+representations%2E+In+Proc%2E+of+CoNLL%2E)
25. Phoebe Mulcaire, Jungo Kasai, and Noah A. Smith. 2019b. Polyglot contextual representations improve crosslingual transfer. In Proc. of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Phoebe+Mulcaire%2C+Jungo+Kasai%2C+and+Noah+A%2E+Smith%2E+2019b%2E+Polyglot+contextual+representations+improve+crosslingual+transfer%2E+In+Proc%2E+of+NAACL%2DHLT%2E)
26. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In Proc. of NAACL-HLT.
27. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. [How multilingual is multilingual BERT?](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/) In Proc. of ACL.
28. Samuel Rönnqvist, Jenna Kanerva, Tapio Salakoski, and Filip Ginter. 2019. Is multilingual BERT fluent in language generation? In Proc. of the First NLPL Workshop on Deep Learning for Natural Language Processing. [find](https://scholar.google.com/scholar?q=Samuel+Ro%CC%88nnqvist%2C+Jenna+Kanerva%2C+Tapio+Salakoski%2C+and+Filip+Ginter%2E+2019%2E+Is+multilingual+BERT+fluent+in+language+generation%3F+In+Proc%2E+of+the+First+NLPL+Workshop+on+Deep+Learning+for+Natural+Language)
29. Tal Schuster, Ori Ram, Regina Barzilay, and Amir Globerson. 2019. [Cross-lingual alignment of contextual word embeddings, with applications to zero-shot dependency parsing](https://md.research.elamur.ai/free/en/papers/schuster-2019-cross-lingual-alignment-contextual-word/). In Proc. of NAACL-HLT.
30. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proc. of ACL.
31. Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. [How to fine-tune BERT for text classification?](https://md.research.elamur.ai/free/en/papers/sun-2019-fine-tune-bert-text-classification/) arXiv:1905.05583 [cs.CL].
32. Peter Trudgill. 2003. A Glossary of Sociolinguistics. Edinburgh University Press. [find](https://scholar.google.com/scholar?q=Peter+Trudgill%2E+2003%2E+A+Glossary+of+Sociolinguistics%2E+Edinburgh+University+Press%2E)
33. Henry Tsai, Jason Riesa, Melvin Johnson, Naveen Arivazhagan, Xin Li, and Amelia Archer. 2019. Small and practical BERT models for sequence labeling. In Proc. of EMNLP-IJCNLP. [find](https://scholar.google.com/scholar?q=Henry+Tsai%2C+Jason+Riesa%2C+Melvin+Johnson%2C+Naveen+Arivazhagan%2C+Xin+Li%2C+and+Amelia+Archer%2E+2019%2E+Small+and+practical+BERT+models+for+sequence+labeling%2E+In+Proc%2E+of+EMNLP%2DIJCNLP%2E)
34. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Proc. of NeurIPS.
35. Hongmin Wang, Jie Yang, and Yue Zhang. 2019. From genesis to creole language: Transfer learning for singlish universal dependencies parsing and pos tagging. ACM Transactions on Asian and Low-Resource Language Information Processing, 19(1). [find](https://scholar.google.com/scholar?q=Hongmin+Wang%2C+Jie+Yang%2C+and+Yue+Zhang%2E+2019%2E+From+genesis+to+creole+language%3A+Transfer+learning+for+singlish+universal+dependencies+parsing+and+pos+tagging%2E+ACM+Transactions+on+Asian+and+Low%2DResource+)
36. Hongmin Wang, Yue Zhang, GuangYong Leonard Chan, Jie Yang, and Hai Leong Chieu. 2017. [Universal dependencies parsing for colloquial Singaporean English](https://md.research.elamur.ai/free/en/papers/wang-2017-universal-dependencies-parsing-colloquial-singaporean/). In Proc. of ACL.
37. Shijie Wu and Mark Dredze. 2019. [Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT](https://md.research.elamur.ai/free/en/papers/wu-2019-beto-bentz-becas-surprising-cross/). In Proc. of EMNLP-IJCNLP.
38. Shijie Wu and Mark Dredze. 2020. [Are all languages created equal in multilingual BERT?](https://md.research.elamur.ai/free/en/papers/wu-2020-are-all-languages-created-equal/) In Proc. of the 5th Workshop on Representation Learning for NLP.
39. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, N… [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V%2E+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+Jeff+Klingner%2C+Apurva+Shah%2C+Melvin+Johnson%2C+Xiaobing+Liu%2C+%C5%81uk)
40. Daniel Zeman, Joakim Nivre, Mitchell Abrams, Noëmi Aepli, Željko Agić, Lars Ahrenberg, Gabrielė Alek-sandravičiūtė, Lene Antonsen, Katya Aplonova, Maria Jesus Aranzabe, Gashaw Arutie, Masayuki Asahara, Luma Ateyah, Mohammed Attia, Aitziber Atutxa, Liesbeth Augustinus, Elena Badmaeva, Miguel B… [find](https://scholar.google.com/scholar?q=Daniel+Zeman%2C+Joakim+Nivre%2C+Mitchell+Abrams%2C+Noe%CC%88mi+Aepli%2C+Z%CC%8Celjko+Agic%CC%81%2C+Lars+Ahrenberg%2C+Gabriele%CC%87+Alek%2Dsandravic%CC%8Ciu%CC%84te%CC%87%2C+Lene+Antonsen%2C+Katya+Aplonova%2C+Maria+Jesus+Aranzabe%2C+Gashaw+Arutie%2C+Masayuki+)
41. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Deniz+Yuret%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2016%2E+Transfer+learning+for+low%2Dresource+neural+machine+translation%2E+In+Proc%2E+of+EMNLP%2E)
42. A Supplementary Material to Accompany Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank This supplement contains further details about the experiments presented in the main paper. A.1 Vocabulary Augmentation and Statistics Language GA MT SING VI Original 12807 49791 3 6955 Augment… [find](https://scholar.google.com/scholar?q=A+Supplementary+Material+to+Accompany+Parsing+with+Multilingual+BERT%2C+a+Small+Corpus%2C+and+a+Small+Treebank+This+supplement+contains+further+details+about+the+experiments+presented+in+the+main+paper%2E+A)

*Updated 24 October 2026*