---
title: "TransMI — paper by Liu et al. (2025) — summary and ideas"
description: "The authors propose TransMI (Transliterate-Merge-Initialize), a procedure that requires no training… 48 references, cited by 10 papers here."
authors: ["Yihong Liu", "Chunlan Ma", "Haotian Ye", "Hinrich Schütze"]
year: 2025
venue: "Proceedings of the 31st International Conference on Computational Linguistics"
source: "https://aclanthology.org/2025.coling-main.32/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2025-transmi-framework-create-strong-baselines/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2025-transmi-framework-create-strong-baselines/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/) / TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data

# TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data

**Authors:** [Yihong Liu](https://md.research.elamur.ai/free/en/authors/yihong-liu/), [Chunlan Ma](https://md.research.elamur.ai/free/en/authors/chunlan-ma/), [Haotian Ye](https://md.research.elamur.ai/free/en/authors/haotian-ye/) and [Hinrich Schütze](https://md.research.elamur.ai/free/en/authors/hinrich-schutze/) · **Year:** 2025 · **Venue:** [Proceedings of the 31st International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2025](https://md.research.elamur.ai/free/en/venues/coling/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.coling-main.32/) · [PDF](https://aclanthology.org/2025.coling-main.32.pdf) · [OpenAlex](https://openalex.org/W4397028272)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/liu-2025-transmi-framework-create-strong-baselines/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Writing related languages in one shared script can improve crosslingual transfer. However, existing mPLMs split romanized text into meaningless pieces, so earlier work usually had to pretrain new models or run adaptation training. The authors propose TransMI (Transliterate-Merge-Initialize), a procedure that requires no training and turns an existing mPLM into a baseline for Latin-transliterated input. It works in three steps. First, every subword in the model's SentencePiece Unigram vocabulary is romanized with Uroman. Second, the romanized subwords are added to the tokenizer, using one of three modes to resolve cases where several original subwords share the same Latin form. Third, embeddings for the added subwords are built from the original model's embeddings. The method was applied to three recent strong mPLMs and tested on sentence retrieval, text classification and sequence labeling. The modified models score almost the same as the originals on original-script data, while outperforming them on transliterated data by 3% to 34%. This gives a cheap starting point for processing romanized text and for later fine-tuning or continued pretraining.

## Key points
- **Method:** TransMI romanizes the vocabulary, merges it with the original tokenizer and initializes the new embeddings, all without gradient updates. One merge variant is Max-Merge. The modified tokenizer splits pinyin such as "jintianshigehaotianqi" into real word units.
- **Results:** Improvements on transliterated evaluation range from 3% to 34% across models and tasks. Code and models are released at github.com/cisnlp/TransMI.
- **Gains by script group:** Cyrl, Arab, Deva and Other gain the most. Hani and Latn gain the least. For Hani, the logographic script and the tones lost in romanization cause ambiguity. For Latn, Uroman only strips diacritics.
- **Remaining gap:** Performance on transliterated data still trails original-script performance, for two reasons:
  - Transliteration ambiguity: "shiwu" stands for both 食物 and 时务, and "miso" matches both a Greek and a Japanese word.
  - Altered tokenization: on SR-B, FURINA (Max-Merge) yields shorter sequences on transliterated text than the original FURINA tokenizer.
- **Limitations:**
  - Only Unigram-based tokenizers and the Uroman tool were tested.
  - The authors suggest WordPiece models such as mBERT could be handled by using subword frequencies instead of log probabilities.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 107 works · 56,478 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works

## Cite

Cite as: Yihong Liu, Chunlan Ma, Haotian Ye and Hinrich Schütze. 2025. TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2025transmi,
  title = {{TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data}},
  author = {Yihong Liu, Chunlan Ma, Haotian Ye and Hinrich Schütze},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.32/}
}
```

- [← previous Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs](https://md.research.elamur.ai/free/en/papers/mahfuz-2025-too-late-train-too-early/)
- [next → Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models](https://md.research.elamur.ai/free/en/papers/vazhentsev-2025-unconditional-truthfulness-learning-unconditional-uncertainty/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/)

## References

48 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba Alabi, Yanke Mao, Haonan Gao, and En-Shiun Lee. 2024. [SIB-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects](https://md.research.elamur.ai/free/en/papers/adelani-2023-sib-200-simple-inclusive-big/). In Proceedings of the 18th Conference of the European Chapter of…
2. Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow. 2022. [Adapting pre-trained language models to African languages via multilingual adaptive fine-tuning](https://md.research.elamur.ai/free/en/papers/alabi-2022-adapting-pre-trained-language-models/). In Proceedings of the 29th International Conference on Computational Linguistics, pages 4336–4349, Gyeongju, Republic…
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
4. Tejas Dhamecha, Rudra Murthy, Samarth Bharadwaj, Karthik Sankaranarayanan, and Pushpak Bhattacharyya. 2021. Role of Language Relatedness in Multilingual Fine-tuning of Language Models: A Case Study in Indo-Aryan Languages. In Proceedings of the 2021 Conference on Empirical Methods in Natural Languag… [find](https://scholar.google.com/scholar?q=Tejas+Dhamecha%2C+Rudra+Murthy%2C+Samarth+Bharadwaj%2C+Karthik+Sankaranarayanan%2C+and+Pushpak+Bhattacharyya%2E+2021%2E+Role+of+Language+Relatedness+in+Multilingual+Fine%2Dtuning+of+Language+Models%3A+A+Case+Study+in)
5. Chantal Amrhein and Rico Sennrich. 2020. On Romanization for model transfer between scripts in neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2461–2469, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chantal+Amrhein+and+Rico+Sennrich%2E+2020%2E+On+Romanization+for+model+transfer+between+scripts+in+neural+machine+translation%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2C+pag)
6. Philip Gage. 1994. A new algorithm for data compression. The C Users Journal, 12(2):23–38. [find](https://scholar.google.com/scholar?q=Philip+Gage%2E+1994%2E+A+new+algorithm+for+data+compression%2E+The+C+Users+Journal%2C+12%282%29%3A23%E2%80%9338%2E)
7. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. [On the cross-lingual transferability of monolingual representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online. Association for Computational Linguistics.
8. Vikrant Goyal, Sourav Kumar, and Dipti Misra Sharma. 2020. Efficient neural machine translation for low-resource languages via exploiting related languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 162–168, Online. A… [find](https://scholar.google.com/scholar?q=Vikrant+Goyal%2C+Sourav+Kumar%2C+and+Dipti+Misra+Sharma%2E+2020%2E+Efficient+neural+machine+translation+for+low%2Dresource+languages+via+exploiting+related+languages%2E+In+Proceedings+of+the+58th+Annual+Meeting+o)
9. Mikel Artetxe and Holger Schwenk. 2019. [Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond](https://md.research.elamur.ai/free/en/papers/artetxe-2018-massively-multilingual-sentence-embeddings-zero/). Transactions of the Association for Computational Linguistics, 7:597–610.
10. Ethan C. Chau and Noah A. Smith. 2021. Specializing multilingual language models: An empirical study. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 51–61, Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ethan+C%2E+Chau+and+Noah+A%2E+Smith%2E+2021%2E+Specializing+multilingual+language+models%3A+An+empirical+study%2E+In+Proceedings+of+the+1st+Workshop+on+Multilingual+Representation+Learning%2C+pages+51%E2%80%9361%2C+Punta+Can)
11. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlingu)
12. Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. 2021. [Universal Dependencies](https://md.research.elamur.ai/free/en/papers/marneffe-2021-universal-dependencies-2/). Computational Linguistics, 47(2):255–308.
13. Arthur P Dempster, Nan M Laird, and Donald B Rubin. 1977. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological), 39(1):1–22. [find](https://scholar.google.com/scholar?q=Arthur+P+Dempster%2C+Nan+M+Laird%2C+and+Donald+B+Rubin%2E+1977%2E+Maximum+likelihood+from+incomplete+data+via+the+em+algorithm%2E+Journal+of+the+royal+statistical+society%3A+series+B+%28methodological%29%2C+39%281%29%3A1%E2%80%9322%2E)
14. Mozhdeh Gheini and Jonathan May. 2019. A universal parent model for low-resource neural machine translation transfer. arXiv preprint arXiv:1909.06516. — [arXiv:1909.06516](https://arxiv.org/abs/1909.06516) [find](https://scholar.google.com/scholar?q=Mozhdeh+Gheini+and+Jonathan+May%2E+2019%2E+A+universal+parent+model+for+low%2Dresource+neural+machine+translation+transfer%2E+arXiv+preprint+arXiv%3A1909%2E06516%2E+%E2%80%94+%5BarXiv%3A1909%2E06516%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1909%2E0)
15. Michael A. Hedderich, Lukas Lange, Heike Adel, Jannik Strötgen, and Dietrich Klakow. 2021. [A survey on recent approaches for natural language processing in low-resource scenarios](https://md.research.elamur.ai/free/en/papers/hedderich-2021-survey-recent-approaches-natural-language/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Hu…
16. Ulf Hermjakob, Jonathan May, and Kevin Knight. 2018. Out-of-the-box universal Romanization tool uroman. In Proceedings of ACL 2018, System Demonstrations, pages 13–18, Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ulf+Hermjakob%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2018%2E+Out%2Dof%2Dthe%2Dbox+universal+Romanization+tool+uroman%2E+In+Proceedings+of+ACL+2018%2C+System+Demonstrations%2C+pages+13%E2%80%9318%2C+Melbourne%2C+Australia%2E+Associatio)
17. Valentin Hofmann, Hinrich Schuetze, and Janet Pierrehumbert. 2022. An embarrassingly simple method to mitigate undesirable properties of pretrained language model tokenizers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 38… [find](https://scholar.google.com/scholar?q=Valentin+Hofmann%2C+Hinrich+Schuetze%2C+and+Janet+Pierrehumbert%2E+2022%2E+An+embarrassingly+simple+method+to+mitigate+undesirable+properties+of+pretrained+language+model+tokenizers%2E+In+Proceedings+of+the+60t)
18. Ayyoob ImaniGooghari, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André Martins, François Yvon, and Hinrich Schütze. 2023. Glot500: Scaling multilingual corpora and language models to 500 languages. In Proceedings of the 61st Choo… [find](https://scholar.google.com/scholar?q=Ayyoob+ImaniGooghari%2C+Peiqin+Lin%2C+Amir+Hossein+Kargaran%2C+Silvia+Severini%2C+Masoud+Jalili+Sabet%2C+Nora+Kassner%2C+Chunlan+Ma%2C+Helmut+Schmid%2C+Andr%C3%A9+Martins%2C+Fran%C3%A7ois+Yvon%2C+and+Hinrich+Sch%C3%BCtze%2E+2023%2E+Glot500)
19. Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1082– 1117, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers%29%2C+pages+1082%E2%80%93+1117%2C+Toronto%2C+Canada%2E+Association+for+Computational+Linguistics%2E)
20. Jaavid J, Raj Dabre, Aswanth M, Jay Gala, Thanmay Jayakumar, Ratish Puduppully, and Anoop Kunchukuttan. 2024. RomanSetu: Efficiently unlocking multilingual capabilities of large language models via Romanization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Jaavid+J%2C+Raj+Dabre%2C+Aswanth+M%2C+Jay+Gala%2C+Thanmay+Jayakumar%2C+Ratish+Puduppully%2C+and+Anoop+Kunchukuttan%2E+2024%2E+RomanSetu%3A+Efficiently+unlocking+multilingual+capabilities+of+large+language+models+via+Ro)
21. Siyang Liu, Naihao Deng, Sahand Sabour, Yilin Jia, Minlie Huang, and Rada Mihalcea. 2023. Task-adaptive tokenization: Enhancing long-form text generation efficacy in mental health and beyond. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15264– 1528… [find](https://scholar.google.com/scholar?q=Siyang+Liu%2C+Naihao+Deng%2C+Sahand+Sabour%2C+Yilin+Jia%2C+Minlie+Huang%2C+and+Rada+Mihalcea%2E+2023%2E+Task%2Dadaptive+tokenization%3A+Enhancing+long%2Dform+text+generation+efficacy+in+mental+health+and+beyond%2E+In+Proce)
22. Masoud Jalili Sabet, Philipp Dufter, François Yvon, and Hinrich Schütze. 2020. SimAlign: High quality word alignments without parallel training data using static and contextualized embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1627–1643, Online. Associat… [find](https://scholar.google.com/scholar?q=Masoud+Jalili+Sabet%2C+Philipp+Dufter%2C+Fran%C3%A7ois+Yvon%2C+and+Hinrich+Sch%C3%BCtze%2E+2020%2E+SimAlign%3A+High+quality+word+alignments+without+parallel+training+data+using+static+and+contextualized+embeddings%2E+In+Find)
23. Yihong Liu, Peiqin Lin, Mingyang Wang, and Hinrich Schuetze. 2024a. OFA: A framework of initializing unseen subword embeddings for efficient large-scale multilingual continued pretraining. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1067–1097, Mexico City, Mexico.… [find](https://scholar.google.com/scholar?q=Yihong+Liu%2C+Peiqin+Lin%2C+Mingyang+Wang%2C+and+Hinrich+Schuetze%2E+2024a%2E+OFA%3A+A+framework+of+initializing+unseen+subword+embeddings+for+efficient+large%2Dscale+multilingual+continued+pretraining%2E+In+Findings)
24. Teruno Kajiura, Shiho Takano, Tatsuya Hiraoka, and Kimio Kuramitsu. 2023. Vocabulary replacement in SentencePiece for domain adaptation. In Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation, pages 645–652, Hong Kong, China. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Teruno+Kajiura%2C+Shiho+Takano%2C+Tatsuya+Hiraoka%2C+and+Kimio+Kuramitsu%2E+2023%2E+Vocabulary+replacement+in+SentencePiece+for+domain+adaptation%2E+In+Proceedings+of+the+37th+Pacific+Asia+Conference+on+Language%2C)
25. Yihong Liu, Chunlan Ma, Haotian Ye, and Hinrich Schuetze. 2024b. TransliCo: A contrastive learning framework to address the script barrier in multilingual pretrained language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p… [find](https://scholar.google.com/scholar?q=Yihong+Liu%2C+Chunlan+Ma%2C+Haotian+Ye%2C+and+Hinrich+Schuetze%2E+2024b%2E+TransliCo%3A+A+contrastive+learning+framework+to+address+the+script+barrier+in+multilingual+pretrained+language+models%2E+In+Proceedings+of)
26. Diederik P. Kingma and Jimmy Ba. 2015. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
27. Taku Kudo. 2018. [Subword regularization: Improving neural network translation models with multiple subword candidates](https://md.research.elamur.ai/free/en/papers/kudo-2018-subword-regularization-improving-neural-network/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 66–75, Melbourne, Australia. Association for Computational…
28. Taku Kudo and John Richardson. 2018. [SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing](https://md.research.elamur.ai/free/en/papers/kudo-2018-sentencepiece-simple-language-independent-subword/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71, Brussels, Belgium. Associ…
29. Anastasios Lamproudis, Aron Henriksson, and Hercules Dalianis. 2022. Vocabulary modifications for domain-adaptive pretraining of clinical language models. In HEALTHINF, pages 180–188. [find](https://scholar.google.com/scholar?q=Anastasios+Lamproudis%2C+Aron+Henriksson%2C+and+Hercules+Dalianis%2E+2022%2E+Vocabulary+modifications+for+domain%2Dadaptive+pretraining+of+clinical+language+models%2E+In+HEALTHINF%2C+pages+180%E2%80%93188%2E)
30. Peiqin Lin, Shaoxiong Ji, Jörg Tiedemann, André FT Martins, and Hinrich Schütze. 2024. [Mala-500: Massive language adaptation of large language models](https://md.research.elamur.ai/free/en/papers/lin-2024-mala-500-massive-language-adaptation/). arXiv preprint arXiv:2401.13303.
31. Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, and Graham Neubig. 2019. [find](https://scholar.google.com/scholar?q=Yu%2DHsiang+Lin%2C+Chian%2DYu+Chen%2C+Jean+Lee%2C+Zirui+Li%2C+Yuyan+Zhang%2C+Mengzhou+Xia%2C+Shruti+Rijhwani%2C+Junxian+He%2C+Zhisong+Zhang%2C+Xuezhe+Ma%2C+Antonios+Anastasopoulos%2C+Patrick+Littell%2C+and+Graham+Neubig%2E+2019%2E)
32. Yihong Liu, Mingyang Wang, Amir Hossein Kargaran, Ayyoob Imani, Orgest Xhelili, Haotian Ye, Chunlan Ma, François Yvon, and Hinrich Schütze. 2024c. [How transliterations improve crosslingual alignment](https://md.research.elamur.ai/free/en/papers/liu-2025-how-transliterations-improve-crosslingual-alignment/). arXiv preprint arXiv:2409.17326.
33. Zihan Liu. 2022. Effective Transfer Learning for Low-Resource Natural Language Understanding. Hong Kong University of Science and Technology (Hong Kong). [find](https://scholar.google.com/scholar?q=Zihan+Liu%2E+2022%2E+Effective+Transfer+Learning+for+Low%2DResource+Natural+Language+Understanding%2E+Hong+Kong+University+of+Science+and+Technology+%28Hong+Kong%29%2E)
34. Chunlan Ma, Ayyoob ImaniGooghari, Haotian Ye, Ehsaneddin Asgari, and Hinrich Schütze. 2023. Taxi1500: A multilingual dataset for text classification in 1500 languages. arXiv preprint arXiv:2305.08487. — [arXiv:2305.08487](https://arxiv.org/abs/2305.08487) [find](https://scholar.google.com/scholar?q=Chunlan+Ma%2C+Ayyoob+ImaniGooghari%2C+Haotian+Ye%2C+Ehsaneddin+Asgari%2C+and+Hinrich+Sch%C3%BCtze%2E+2023%2E+Taxi1500%3A+A+multilingual+dataset+for+text+classification+in+1500+languages%2E+arXiv+preprint+arXiv%3A2305%2E08487%2E)
35. Chunlan Ma, Yihong Liu, Haotian Ye, and Hinrich Schütze. 2024. Exploring the role of transliteration in in-context learning for low-resource languages written in non-latin scripts. arXiv preprint arXiv:2407.02320. [find](https://scholar.google.com/scholar?q=Chunlan+Ma%2C+Yihong+Liu%2C+Haotian+Ye%2C+and+Hinrich+Sch%C3%BCtze%2E+2024%2E+Exploring+the+role+of+transliteration+in+in%2Dcontext+learning+for+low%2Dresource+languages+written+in+non%2Dlatin+scripts%2E+arXiv+preprint+arXi)
36. Alexandre Magueresse, Vincent Carles, and Evan Heetderks. 2020. Low-resource languages: A review of past work and future challenges. arXiv preprint arXiv:2006.07264. — [arXiv:2006.07264](https://arxiv.org/abs/2006.07264) [find](https://scholar.google.com/scholar?q=Alexandre+Magueresse%2C+Vincent+Carles%2C+and+Evan+Heetderks%2E+2020%2E+Low%2Dresource+languages%3A+A+review+of+past+work+and+future+challenges%2E+arXiv+preprint+arXiv%3A2006%2E07264%2E+%E2%80%94+%5BarXiv%3A2006%2E07264%5D%28https%3A%2F%2Farxiv)
37. Ibraheem Muhammad Moosa, Mahmud Elahi Akhter, and Ashfia Binte Habib. 2023. Does transliteration help multilingual language modeling? In Findings of the Association for Computational Linguistics: EACL 2023, pages 670–685, Dubrovnik, Croatia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ibraheem+Muhammad+Moosa%2C+Mahmud+Elahi+Akhter%2C+and+Ashfia+Binte+Habib%2E+2023%2E+Does+transliteration+help+multilingual+language+modeling%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EACL)
38. Benjamin Muller, Antonios Anastasopoulos, Benoît Sagot, and Djamé Seddah. 2021. [When being unseen from mBERT is just the beginning: Handling new languages with multilingual language models](https://md.research.elamur.ai/free/en/papers/muller-2021-when-being-unseen-mbert-is/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Lingu…
39. Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017. [Cross-lingual name tagging and linking for 282 languages](https://md.research.elamur.ai/free/en/papers/pan-2017-cross-lingual-name-tagging-linking/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1946–1958, Vancouver, Canada. A…
40. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2021. [UNKs everywhere: Adapting multilingual language models to new scripts](https://md.research.elamur.ai/free/en/papers/pfeiffer-2021-unks-everywhere-adapting-multilingual-language/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10186–10203, Online and Punta Cana, Dominican Republic. Asso…
41. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. [How multilingual is multilingual BERT?](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/) In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001, Florence, Italy. Association for Computational Linguistics.
42. Sukannya Purkayastha, Sebastian Ruder, Jonas Pfeiffer, Iryna Gurevych, and Ivan Vulić. 2023. Romanization-based large-scale adaptation of multilingual language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7996–8005, Singapore. Association for Computational… [find](https://scholar.google.com/scholar?q=Sukannya+Purkayastha%2C+Sebastian+Ruder%2C+Jonas+Pfeiffer%2C+Iryna+Gurevych%2C+and+Ivan+Vulic%CC%81%2E+2023%2E+Romanization%2Dbased+large%2Dscale+adaptation+of+multilingual+language+models%2E+In+Findings+of+the+Association+)
43. Vin Sachidananda, Jason Kessler, and Yi-An Lai. 2021. Efficient domain adaptation of language models via adaptive tokenization. In Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing, pages 155–165, Virtual. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Vin+Sachidananda%2C+Jason+Kessler%2C+and+Yi%2DAn+Lai%2E+2021%2E+Efficient+domain+adaptation+of+language+models+via+adaptive+tokenization%2E+In+Proceedings+of+the+Second+Workshop+on+Simple+and+Efficient+Natural+La)
44. Mike Schuster and Kaisuke Nakajima. 2012. Japanese and korean voice search. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2012, Kyoto, Japan, March 25-30, 2012, pages 5149–5152. IEEE. [find](https://scholar.google.com/scholar?q=Mike+Schuster+and+Kaisuke+Nakajima%2E+2012%2E+Japanese+and+korean+voice+search%2E+In+2012+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing%2C+ICASSP+2012%2C+Kyoto%2C+Japan%2C+March+25%2D30%2C+20)
45. Shijie Wu and Mark Dredze. 2019. [Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT](https://md.research.elamur.ai/free/en/papers/wu-2019-beto-bentz-becas-surprising-cross/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 833–844,…
46. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+et+al%2E+2016%2E+Google%E2%80%99s+neural+machine+translation+system%3A+Bridg)
47. Orgest Xhelili, Yihong Liu, and Hinrich Schuetze. 2024. Breaking the script barrier in multilingual pre-trained language models with transliteration-based post-training alignment. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11283–11296, Miami, Florida, USA. Associ… [find](https://scholar.google.com/scholar?q=Orgest+Xhelili%2C+Yihong+Liu%2C+and+Hinrich+Schuetze%2E+2024%2E+Breaking+the+script+barrier+in+multilingual+pre%2Dtrained+language+models+with+transliteration%2Dbased+post%2Dtraining+alignment%2E+In+Findings+of+the+A)
48. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1568–1575, Austin, Texas. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Deniz+Yuret%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2016%2E+Transfer+learning+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Lan)

*Updated 30 October 2026*