---
title: "UM-DFKI Maltese Speech Translation — Williams 2023"
description: "The paper describes the University of Malta and DFKI submission to the first IWSLT Maltese→English speech… 0.6 BLEU. 34 references, cited by 8 papers here."
authors: ["Aiden Williams", "Kurt Abela", "Rishu Kumar", "Martin Bär", "Hannah Billinghurst", "Kurt Micallef", "Ahnaf Mozib Samin", "Andrea DeMarco", "Lonneke van der Plas", "Claudia Borg"]
year: 2023
venue: "Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023)"
source: "https://aclanthology.org/2023.iwslt-1.41/"
doi: "10.18653/v1/2023.iwslt-1.41"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/williams-2023-um-dfki-maltese-speech-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/williams-2023-um-dfki-maltese-speech-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) / UM-DFKI Maltese Speech Translation

# UM-DFKI Maltese Speech Translation

**Authors:** [Aiden Williams](https://md.research.elamur.ai/free/en/authors/aiden-williams/), [Kurt Abela](https://md.research.elamur.ai/free/en/authors/kurt-abela/), Rishu Kumar, Martin Bär, Hannah Billinghurst, [Kurt Micallef](https://md.research.elamur.ai/free/en/authors/kurt-micallef/), [Ahnaf Mozib Samin](https://md.research.elamur.ai/free/en/authors/ahnaf-mozib-samin/), [Andrea DeMarco](https://md.research.elamur.ai/free/en/authors/andrea-demarco/), [Lonneke van der Plas](https://md.research.elamur.ai/free/en/authors/lonneke-van-der-plas/) and [Claudia Borg](https://md.research.elamur.ai/free/en/authors/claudia-borg/) · **Year:** 2023 · **Venue:** Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.iwslt-1.41/) · [PDF](https://aclanthology.org/2023.iwslt-1.41.pdf) · [DOI 10.18653/v1/2023.iwslt-1.41](https://doi.org/10.18653/v1/2023.iwslt-1.41) · [OpenAlex](https://openalex.org/W4385570111)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/williams-2023-um-dfki-maltese-speech-translation/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper describes the University of Malta and DFKI submission to the first IWSLT Maltese→English speech translation task, which is a low-resource setting. The team used a cascade design. An ASR stage, built on the 300M-parameter XLS-R model, produced Maltese transcripts, and a separately trained mBART-50 model translated them into English. The two stages were trained independently and evaluated together. The ASR training data was about 50 hours of annotated Maltese speech drawn from MASRI, CommonVoice and parliamentary recordings. The authors also tested whether mixing in 50-hour CommonVoice subsets of Italian, French or Arabic helps, which gave five ASR variants. The primary system, fine-tuned on Maltese only, reached 0.6 BLEU, so absolute quality was very poor. A qualitative error analysis traces the failures to three sources: lost Maltese-specific letters, phonetically plausible misrecognitions, and a mismatch between formal legal training text and informal podcast speech. The work matters mainly as a documented baseline and as a source of diagnostic lessons for low-resource speech translation of languages that pretrained models do not cover.

## Key points
- The cascade pairs XLS-R 300M (ASR) with mBART-50 (MT). Neither mBART-25 nor mBART-50 saw Maltese during pretraining.
- ASR fine-tuning used the Huggingface Trainer with these settings:
  - 30 epochs
  - AdamW with a learning rate of 3e-4
  - 500 warm-up steps
  - gradient accumulation that effectively quadruples the batch size
- Five ASR variants were trained: MT Only, MT+AR, MT+FR, MT+IT and MT+All. Official BLEU scores were 0.6, 0.4, 0.3, 0.4 and 0.7 respectively. MT Only was chosen as the primary submission.
- The data released for the task was not used, because segmenting it in time proved difficult. The team relied on the earlier dataset from Williams (2022) instead.
- Planned improvements:
  - extend the mBART vocabulary with Maltese tokens
  - back-translate using Korpus Malti v4
  - clean the noisy crawled parallel data
  - try Whisper and MMS
  - transliterate foreign characters
  - add a language model to correct ASR output

## Interesting ideas
- Following one test sentence through every stage showed exactly where meaning broke, for example "merba" (a mishearing of "merħba", welcome) being read as "erba" (four), and "podcast" coming out as "pot kast".
- Adding data from related languages actually hurt, because the models learned those languages' letter inventories and stopped producing Maltese-specific characters; checking the output character set is a cheap diagnostic for this.
- Before mixing foreign data, the authors propose mapping foreign characters onto their target-language equivalents rather than training on raw mixed alphabets.
- The literature review describes a system that passes ASR output to MT as probability vectors instead of a single hard choice, which reduces error propagation in cascades.
- Much of the MT failure came from a domain gap: the parallel data was formal legal text, while the test input was conversational speech.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Evaluation reports could include a per-stage trace of a few failed items, so each wrong answer is pinned to the tokenizer, parser, frame or world step that introduced it.
- **I2** (tokens): When training the tagger or lemmatizer on mixed or out-of-domain data, monitor whether language-specific forms (e.g., rare affixes or spellings) vanish from predictions.
- **I3** (tree): The same principle supports converting other UD dialects or treebank conventions into Mova's own conventions before pooling them for perceptron or gparse training.
- **I4** (frames): The frame builder could receive several scored parse alternatives instead of a single tree, and keep track of which reading each frame came from so the choice stays explainable.
- **I5** (learn): Silver data and evaluation sets could be split by genre (fables, school tests, dialogue), so that domain gaps show up as separate scores instead of being averaged away.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/)

Topics: [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/)

## Institutions

- [Idiap Research Institute](https://md.research.elamur.ai/free/en/labs/idiap-research-institute/) — Research institute, Martigny-Combe, Switzerland · 17 works · 1,669 citations
- [University of Malta](https://md.research.elamur.ai/free/en/universities/university-of-malta/) — University, Msida, Malta · 12 works · 573 citations

## Cite

Cite as: Aiden Williams, Kurt Abela, Rishu Kumar, Martin Bär, Hannah Billinghurst, Kurt Micallef, Ahnaf Mozib Samin, Andrea DeMarco, Lonneke van der Plas and Claudia Borg. 2023. UM-DFKI Maltese Speech Translation. Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{williams2023um,
  title = {{UM-DFKI Maltese Speech Translation}},
  author = {Aiden Williams, Kurt Abela, Rishu Kumar, Martin Bär, Hannah Billinghurst, Kurt Micallef, Ahnaf Mozib Samin, Andrea DeMarco, Lonneke {van der Plas} and Claudia Borg},
  booktitle = {Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023)},
  year = {2023},
  doi = {10.18653/v1/2023.iwslt-1.41},
  url = {https://doi.org/10.18653/v1/2023.iwslt-1.41}
}
```

- [← previous Optimizing the Size of Subword Vocabularies in Dialect Classification](https://md.research.elamur.ai/free/en/papers/kanjirangat-2023-optimizing-size-subword-vocabularies-dialect/)
- [next → AmQA: Amharic Question Answering Dataset](https://md.research.elamur.ai/free/en/papers/abedissa-2023-amqa-amharic-question-answering-dataset/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/)

## References

34 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Milind Agarwal, Sweta Agrawal, Antonios Anastasopoulos, Ondřej Bojar, Claudia Borg, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, Mingda Chen, William Chen, Khalid Choukri, Alexandra Chronopoulou, Anna Currey, Thierry Declerck, Qianqian Dong, Yannick Estève, Kevin Duh, Marcello Federico, Souhir Gahbiche, Barry Haddow, Benjamin Hsu, Phu Mon Htut, Hirofumi Inaguma, Dávid Javorský, John Judge, Yasumasa Kano, Tom Ko, Rishu Kumar, Pengwei Li, Xutail Ma, Prashant Mathur, Evgeny Matusov, Paul McNamee, John P. McCrae, Kenton Murray, Maria Nadejde, Satoshi Nakamura, Matteo Negri, Ha Nguyen, Jan Niehues, Xing Niu, Atul Ojha Kr., John E. Ortega, Proyag Pal, Juan Pino, Lonneke van der Plas, Peter Polák, Elijah Rippeth, Elizabeth Salesky, Jiatong Shi, Matthias Sperber, Sebastian Stüker, Katsuhito Sudoh, Yun Tang, Brian Thompson, Kevin Tran, Marco Turchi, Alex Waibel, Mingxuan Wang, Shinji Watanabe, and Rodolfo Zevallos. 2023. Findings of the IWSLT 2023 Evaluation Campaign. In Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023). Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/agarwal-2023-findings-iwslt-2023-evaluation-campaign/)
2. Antonios Anastasopoulos, Loïc Barrault, Luisa Bentivogli, Marcely Zanon Boito, Ondřej Bojar, Roldano Cattoni, Anna Currey, Georgiana Dinu, Kevin Duh, Maha Elbayad, Clara Emmanuel, Yannick Estève, Marcello Federico, Christian Federmann, Souhir Gahbiche, Hongyu Gong, Roman Grundkiewicz, Barry Haddow, Benjamin Hsu, Dávid Javorský, Vĕra Kloudová, Surafel Lakew, Xutai Ma, Prashant Mathur, Paul McNamee, Kenton Murray, Maria Nǎdejde, Satoshi Nakamura, Matteo Negri, Jan Niehues, Xing Niu, John Ortega, Juan Pino, Elizabeth Salesky, Jiatong Shi, Matthias Sperber, Sebastian Stüker, Katsuhito Sudoh, Marco Turchi, Yogesh Virkar, Alexander Waibel, Changhan Wang, and Shinji Watanabe. 2022. Findings of the IWSLT 2022 Evaluation Campaign. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 98–157, Dublin, Ireland (in-person and online). Association for Computational Linguistics. [https://mundus-web.coli.uni-saarland.de/](https://mundus-web.coli.uni-saarland.de/) [↗](https://mundus-web.coli.uni-saarland.de/) [our card](https://md.research.elamur.ai/free/en/papers/anastasopoulos-2022-findings-iwslt-2022-evaluation-campaign/)
3. Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. Common voice: A massively-multilingual speech corpus. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4218–4222, Marseille, France. European Language Resources Association. [our card](https://md.research.elamur.ai/free/en/papers/ardila-2019-common-voice-massively-multilingual-speech/)
4. Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, Alexei Baevski, Alexis Conneau, and Michael Auli. 2021. XLS-R: self-supervised cross-lingual speech representation learning at scale. CoRR, abs/2111.09296. [↗](https://arxiv.org/abs/2111.09296) [our card](https://md.research.elamur.ai/free/en/papers/babu-2021-xls-r-self-supervised-cross/)
5. Parnia Bahar, Patrick Wilken, Mattia A. Di Gangi, and Evgeny Matusov. 2021. Without Further Ado: Direct and Simultaneous Speech Translation by AppTek in 2021. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021), pages 52–63, Bangkok, Thailand (online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Parnia+Bahar%2C+Patrick+Wilken%2C+Mattia+A%2E+Di+Gangi%2C+and+Evgeny+Matusov%2E+2021%2E+Without+Further+Ado%3A+Direct+and+Simultaneous+Speech+Translation+by+AppTek+in+2021%2E+In+Proceedings+of+the+18th+International+)
6. Marta Bañón, Miquel Esplà-Gomis, Mikel L. Forcada, Cristian García-Romero, Taja Kuzman, Nikola Ljubešić, Rik van Noord, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Peter Rupnik, Vít Suchomel, Antonio Toral, Tobias van der Werff, and Jaume Zaragoza. 2022. MaCoCu: Massive collection and curation of monolingual and bilingual data: focus on under-resourced languages. In Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, pages 303–304, Ghent, Belgium. European Association for Machine Translation. [find](https://scholar.google.com/scholar?q=Marta+Ba%C3%B1%C3%B3n%2C+Miquel+Espl%C3%A0%2DGomis%2C+Mikel+L%2E+Forcada%2C+Cristian+Garc%C3%ADa%2DRomero%2C+Taja+Kuzman%2C+Nikola+Ljube%C5%A1ic%CC%81%2C+Rik+van+Noord%2C+Leopoldo+Pla+Sempere%2C+Gema+Ram%C3%ADrez%2DS%C3%A1nchez%2C+Peter+Rupnik%2C+V%C3%ADt+Suchomel%2C+Antonio)
7. Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. 2021. Unsupervised Cross-Lingual Representation Learning for Speech Recognition. In Proc. Interspeech 2021, pages 2426–2430. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Alexei+Baevski%2C+Ronan+Collobert%2C+Abdelrahman+Mohamed%2C+and+Michael+Auli%2E+2021%2E+Unsupervised+Cross%2DLingual+Representation+Learning+for+Speech+Recognition%2E+In+Proc%2E+Interspeech+2021%2C+page)
8. Pavel Denisov, Manuel Mager, and Ngoc Thang Vu. 2021. IMS’ Systems for the IWSLT 2021 Low-Resource Speech Translation Task. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021), pages 175–181, Bangkok, Thailand (online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Pavel+Denisov%2C+Manuel+Mager%2C+and+Ngoc+Thang+Vu%2E+2021%2E+IMS%E2%80%99+Systems+for+the+IWSLT+2021+Low%2DResource+Speech+Translation+Task%2E+In+Proceedings+of+the+18th+International+Conference+on+Spoken+Language+Trans)
9. Liang Ding and Dacheng Tao. 2021. The USYD-JD Speech Translation System for IWSLT2021. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021), pages 182–191, Bangkok, Thailand (online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Liang+Ding+and+Dacheng+Tao%2E+2021%2E+The+USYD%2DJD+Speech+Translation+System+for+IWSLT2021%2E+In+Proceedings+of+the+18th+International+Conference+on+Spoken+Language+Translation+%28IWSLT+2021%29%2C+pages+182%E2%80%93191%2C+B)
10. Mark JF Gales, Kate M Knill, Anton Ragni, and Shakti P Rath. 2014. Speech recognition and keyword spotting for low-resource languages: Babel project research at cued. In Fourth International workshop on spoken language technologies for under-resourced languages (SLTU-2014), pages 16–23. International Speech Communication Association (ISCA). [find](https://scholar.google.com/scholar?q=Mark+JF+Gales%2C+Kate+M+Knill%2C+Anton+Ragni%2C+and+Shakti+P+Rath%2E+2014%2E+Speech+recognition+and+keyword+spotting+for+low%2Dresource+languages%3A+Babel+project+research+at+cued%2E+In+Fourth+International+workshop+)
11. Nizar Habash, Nasser Zalmout, Dima Taji, Hieu Hoang, and Maverick Alzate. 2017. A parallel corpus for evaluating machine translation between arabic and european languages. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 235–241. [find](https://scholar.google.com/scholar?q=Nizar+Habash%2C+Nasser+Zalmout%2C+Dima+Taji%2C+Hieu+Hoang%2C+and+Maverick+Alzate%2E+2017%2E+A+parallel+corpus+for+evaluating+machine+translation+between+arabic+and+european+languages%2E+In+Proceedings+of+the+15th+C)
12. Najeh Hajlaoui, David Kolovratnik, Jaakko Väyrynen, Ralf Steinberger, and Daniel Varga. 2014. Dcep-digital corpus of the european parliament. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14). [find](https://scholar.google.com/scholar?q=Najeh+Hajlaoui%2C+David+Kolovratnik%2C+Jaakko+V%C3%A4yrynen%2C+Ralf+Steinberger%2C+and+Daniel+Varga%2E+2014%2E+Dcep%2Ddigital+corpus+of+the+european+parliament%2E+In+Proceedings+of+the+Ninth+International+Conference+on+La)
13. Michael A. Hedderich, Lukas Lange, Heike Adel, Jannik Strötgen, and Dietrich Klakow. 2021. A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2545–2568, Online. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/hedderich-2021-survey-recent-approaches-natural-language/)
14. Carlos Daniel Hernandez Mena, Albert Gatt, Andrea DeMarco, Claudia Borg, Lonneke van der Plas, Amanda Muscat, and Ian Padovani. 2020. MASRI-HEADSET: A Maltese corpus for speech recognition. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 6381–6388, Marseille, France. European Language Resources Association. [our card](https://md.research.elamur.ai/free/en/papers/mena-2020-masri-headset-maltese-corpus-speech/)
15. Diksha Khurana, Aditya Koli, Kiran Khatter, and Sukhdev Singh. 2023. Natural language processing: State of the art, current trends and challenges. Multimedia tools and applications, 82(3):3713–3744. [find](https://scholar.google.com/scholar?q=Diksha+Khurana%2C+Aditya+Koli%2C+Kiran+Khatter%2C+and+Sukhdev+Singh%2E+2023%2E+Natural+language+processing%3A+State+of+the+art%2C+current+trends+and+challenges%2E+Multimedia+tools+and+applications%2C+82%283%29%3A3713%E2%80%933744%2E)
16. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461. [↗](https://arxiv.org/abs/1910.13461) [our card](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
17. Yinglu Li, Minghan Wang, Jiaxin Guo, Xiaosong Qiao, Yuxia Wang, Daimeng Wei, Chang Su, Yimeng Chen, Min Zhang, Shimin Tao, Hao Yang, and Ying Qin. 2022. The HW-TSC’s Offline Speech Translation System for IWSLT 2022 Evaluation. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 239–246, Dublin, Ireland (in-person and online). Association for Computational Linguistics. 2021. Data augmentation for speech recognition in maltese: A low-resource perspective. CoRR, abs/2111.07793. [↗](https://arxiv.org/abs/2111.07793) [our card](https://md.research.elamur.ai/free/en/papers/demarco-2021-analysis-data-augmentation-methods-low/)
18. Kurt Micallef, Albert Gatt, Marc Tanti, Lonneke van der Plas, and Claudia Borg. 2022. Pre-training data quality and quantity for a low-resource language: New corpus and BERT models for Maltese. In Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing, pages 90–101, Hybrid. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kurt+Micallef%2C+Albert+Gatt%2C+Marc+Tanti%2C+Lonneke+van+der+Plas%2C+and+Claudia+Borg%2E+2022%2E+Pre%2Dtraining+data+quality+and+quantity+for+a+low%2Dresource+language%3A+New+corpus+and+BERT+models+for+Maltese%2E+In+Pro)
19. Tuan Nam Nguyen, Thai Son Nguyen, Christian Huber, Ngoc-Quan Pham, Thanh-Le Ha, Felix Schneider, and Sebastian Stüker. 2021. [find](https://scholar.google.com/scholar?q=Tuan+Nam+Nguyen%2C+Thai+Son+Nguyen%2C+Christian+Huber%2C+Ngoc%2DQuan+Pham%2C+Thanh%2DLe+Ha%2C+Felix+Schneider%2C+and+Sebastian+St%C3%BCker%2E+2021%2E)
20. KIT’s IWSLT 2021 Offline Speech Translation System. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021), pages 125–130, Bangkok, Thailand (online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=KIT%E2%80%99s+IWSLT+2021+Offline+Speech+Translation+System%2E+In+Proceedings+of+the+18th+International+Conference+on+Spoken+Language+Translation+%28IWSLT+2021%29%2C+pages+125%E2%80%93130%2C+Bangkok%2C+Thailand+%28online%29%2E+Associat)
21. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations. [our card](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
22. Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. 2020. MLS: A Large-Scale Multilingual Dataset for Speech Research. In Proc. Interspeech 2020, pages 2757–2761. [our card](https://md.research.elamur.ai/free/en/papers/pratap-2020-mls-large-scale-multilingual-dataset/)
23. Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020. Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63(10):1872– 1897. [our card](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/)
24. Ralf Steinberger, Mohamed Ebrahim, Alexandros Poulis, Manuel Carrasco-Benitez, Patrick Schlüter, Marek Przybyszewski, and Signe Gilbro. 2014. An overview of the european union’s highly multilingual parallel corpora. Language resources and evaluation, 48(4):679–707. [find](https://scholar.google.com/scholar?q=Ralf+Steinberger%2C+Mohamed+Ebrahim%2C+Alexandros+Poulis%2C+Manuel+Carrasco%2DBenitez%2C+Patrick+Schl%C3%BCter%2C+Marek+Przybyszewski%2C+and+Signe+Gilbro%2E+2014%2E+An+overview+of+the+european+union%E2%80%99s+highly+multilingual+pa)
25. Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020. Multilingual translation with extensible multilingual pretraining and finetuning. arXiv preprint arXiv:2008.00401. [↗](https://arxiv.org/abs/2008.00401)
26. Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre-training for neural machine translation. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Jiatao+Gu%2C+Naman+Goyal%2C+Xian+Li%2C+Sergey+Edunov%2C+Marjan+Ghazvininejad%2C+Mike+Lewis%2C+and+Luke+Zettlemoyer%2E+2020%2E+Multilingual+denoising+pre%2Dtraining+for+neural+machine+translation%2E)
27. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in opus. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=J%C3%B6rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+opus%2E+In+Proceedings+of+the+Eight+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9912%29%2C+Istanbul%2C+Turkey%2E+European+Langu)
28. Alexandre Magueresse, Vincent Carles, and Evan Heetderks. 2020. Low-resource languages: A review of past work and future challenges. arXiv preprint arXiv:2006.07264. [↗](https://arxiv.org/abs/2006.07264)
29. Jörgen Valk and Tanel Alumäe. 2021. Voxlingua107: A dataset for spoken language recognition. In 2021 IEEE Spoken Language Technology Workshop (SLT), pages 652–658. Carlos Daniel Hernandez Mena, Andrea DeMarco, Claudia Borg, Lonneke van der Plas, and Albert Gatt. [find](https://scholar.google.com/scholar?q=J%C3%B6rgen+Valk+and+Tanel+Alum%C3%A4e%2E+2021%2E+Voxlingua107%3A+A+dataset+for+spoken+language+recognition%2E+In+2021+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2C+pages+652%E2%80%93658%2E+Carlos+Daniel+Hernandez+Mena%2C+Andrea)
30. Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. 2021. VoxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 993–1003, Online. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/wang-2021-voxpopuli-large-scale-multilingual-speech/)
31. Aiden Williams. 2022. The applicability of Wav2Vec 2.0 for low-resource Maltese ASR. B.S. thesis, University of Malta. [find](https://scholar.google.com/scholar?q=Aiden+Williams%2E+2022%2E+The+applicability+of+Wav2Vec+2%2E0+for+low%2Dresource+Maltese+ASR%2E+B%2ES%2E+thesis%2C+University+of+Malta%2E)
32. Marcely Zanon Boito, John Ortega, Hugo Riguidel, Antoine Laurent, Loïc Barrault, Fethi Bougares, Firas Chaabani, Ha Nguyen, Florentin Barbier, Souhir Gahbiche, and Yannick Estève. 2022. ON-TRAC Consortium Systems for the IWSLT 2022 Dialect and Low-resource Speech Translation Tasks. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 308–318, Dublin, Ireland (in-person and online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Marcely+Zanon+Boito%2C+John+Ortega%2C+Hugo+Riguidel%2C+Antoine+Laurent%2C+Lo%C3%AFc+Barrault%2C+Fethi+Bougares%2C+Firas+Chaabani%2C+Ha+Nguyen%2C+Florentin+Barbier%2C+Souhir+Gahbiche%2C+and+Yannick+Est%C3%A8ve%2E+2022%2E+ON%2DTRAC+Consor)
33. Weitai Zhang, Zhongyi Ye, Haitao Tang, Xiaoxi Li, Xinyuan Zhou, Jing Yang, Jianwei Cui, Pan Deng, Mohan Shi, Yifan Song, Dan Liu, Junhua Liu, and Lirong Dai. 2022. The USTC-NELSLIP Offline Speech Translation Systems for IWSLT 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 198–207, Dublin, Ireland (in-person and online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Weitai+Zhang%2C+Zhongyi+Ye%2C+Haitao+Tang%2C+Xiaoxi+Li%2C+Xinyuan+Zhou%2C+Jing+Yang%2C+Jianwei+Cui%2C+Pan+Deng%2C+Mohan+Shi%2C+Yifan+Song%2C+Dan+Liu%2C+Junhua+Liu%2C+and+Lirong+Dai%2E+2022%2E+The+USTC%2DNELSLIP+Offline+Speech+Tran)
34. Ziqiang Zhang and Junyi Ao. 2022. The YiTrans Speech Translation System for IWSLT 2022 Offline Shared Task. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 158–168, Dublin, Ireland (in-person and online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ziqiang+Zhang+and+Junyi+Ao%2E+2022%2E+The+YiTrans+Speech+Translation+System+for+IWSLT+2022+Offline+Shared+Task%2E+In+Proceedings+of+the+19th+International+Conference+on+Spoken+Language+Translation+%28IWSLT+20)