---
title: "Findings of WMT 2024 Shared Task on — Pakray 2024"
description: "This paper reports the results of the WMT 2024 shared task on machine translation for low-resource languages of northeastern India."
authors: ["Partha Pakray", "Santanu Pal", "Advaitha Vetagiri", "Reddi Krishna", "Arnab Kumar Maji", "Sandeep Dash", "Lenin Laitonjam", "Lyngdoh Sarah", "Riyanka Manna"]
year: 2024
venue: "Proceedings of the Ninth Conference on Machine Translation"
source: "https://aclanthology.org/2024.wmt-1.54/"
doi: "10.18653/v1/2024.wmt-1.54"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/pakray-2024-findings-wmt-2024-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/pakray-2024-findings-wmt-2024-shared-task/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2024](https://md.research.elamur.ai/free/papers/2024/page/2/) / Findings of WMT 2024 Shared Task on Low-Resource Indic Languages Translation

# Findings of WMT 2024 Shared Task on Low-Resource Indic Languages Translation

**Authors:** [Partha Pakray](https://md.research.elamur.ai/free/authors/partha-pakray/), [Santanu Pal](https://md.research.elamur.ai/free/authors/santanu-pal/), Advaitha Vetagiri, Reddi Krishna, Arnab Kumar Maji, Sandeep Dash, [Lenin Laitonjam](https://md.research.elamur.ai/free/authors/lenin-laitonjam/), Lyngdoh Sarah, Riyanka Manna · **Year:** 2024 · **Venue:** Proceedings of the Ninth Conference on Machine Translation · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.wmt-1.54/) · [PDF](https://aclanthology.org/2024.wmt-1.54.pdf) · [DOI 10.18653/v1/2024.wmt-1.54](https://doi.org/10.18653/v1/2024.wmt-1.54) · [OpenAlex](https://openalex.org/W4404780636)

**Citations:** 5 citations (openalex-snapshot · 2026-10-14) · 16 in our library, of the 140673 papers whose bibliographies we read

## Summary
This paper reports the results of the WMT 2024 shared task on machine translation for low-resource languages of northeastern India. These languages have few digital resources, little institutional support and complex scripts or morphology. The task follows the 2023 edition and keeps its four pairs: English with Assamese, Mizo, Khasi and Manipuri, in both directions. Training data came from the IndicNE-Corp1.0 parallel and monolingual collection. The organisers built a new test set that is harder than the previous year's. For Khasi, the Sohra (Cherra) dialect was used as the standard variety.

Submissions were scored with five automatic metrics (BLEU, TER, RIBES, METEOR, ChrF) and with human judgements of Adequacy, Fluency and Overall Quality. The most successful approach fine-tuned a large pretrained multilingual model and enlarged the training data with back-translated monolingual text. Scores stayed low overall, which shows these pairs remain hard. The task matters because it offers a shared benchmark for under-supported languages and a basis for growing toward more regional languages.

## Key points
- Eleven teams submitted English–Khasi systems, including DLUT-NLP, HW-TSC, SPRING-IITM, JUNLP and MTNLP-IIITH. Teams could send primary and contrastive runs.
- SPRING-IITM ranked first for English–Mizo and English–Khasi in both directions. Its method was back translation plus fine-tuning NLLB 3.3B, with special tokens and a custom corpus for Khasi.
- HW-TSC was strongest for Manipuri→English. A contrastive run that applied LoRA fine-tuning to a pretrained translation model improved its results further.
- The best English–Khasi runs reached only about 0.12 / 0.10 BLEU (0–1 scale) and about 0.45 / 0.43 ChrF. Their human Overall Quality ratings were close to 5.
- Planned future pairs are English with Nyishi, Bodo, Mising and Kokborok, along with larger corpora.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pakray2024findings,
  title = {{Findings of WMT 2024 Shared Task on Low-Resource Indic Languages Translation}},
  author = {Pakray, Partha and Pal, Santanu and Vetagiri, Advaitha and Krishna, Reddi and Maji, Arnab Kumar and Dash, Sandeep and Laitonjam, Lenin and Sarah, Lyngdoh and Manna, Riyanka},
  booktitle = {Proceedings of the Ninth Conference on Machine Translation},
  year = {2024},
  doi = {10.18653/v1/2024.wmt-1.54},
  url = {https://doi.org/10.18653/v1/2024.wmt-1.54}
}
```

- [← previous Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet](https://md.research.elamur.ai/free/papers/kocmi-2024-findings-wmt24-general-machine-translation/)
- [next → Fine-Tuned Machine Translation Metrics Struggle in Unseen Domains](https://md.research.elamur.ai/free/papers/zouhar-2024-fine-tuned-machine-translation-metrics/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/papers/2024/page/2/)

## References

27 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. James F. Allen. 2003. Natural language processing, page 1218–1222. John Wiley and Sons Ltd., GBR. [find](https://scholar.google.com/scholar?q=James+F%2E+Allen%2E+2003%2E+Natural+language+processing%2C+page+1218%E2%80%931222%2E+John+Wiley+and+Sons+Ltd%2E%2C+GBR%2E)
2. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Ass… [find](https://scholar.google.com/scholar?q=Satanjeev+Banerjee+and+Alon+Lavie%2E+2005%2E+METEOR%3A+An+automatic+metric+for+MT+evaluation+with+improved+correlation+with+human+judgments%2E+In+Proceedings+of+the+ACL+Workshop+on+Intrinsic+and+Extrinsic+Eva) [our card](https://md.research.elamur.ai/free/papers/banerjee-2005-meteor-automatic-metric-mt-evaluation/)
3. Chenfei Ju, Junpeng Liu, Kaiyu Huang, and Degen Huang. 2024. Dlut-nlp machine translation systems for wmt24 low-resource indic language translation. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translation (WMT24), EMNLP, November 15-16, 2024,… [find](https://scholar.google.com/scholar?q=Chenfei+Ju%2C+Junpeng+Liu%2C+Kaiyu+Huang%2C+and+Degen+Huang%2E+2024%2E+Dlut%2Dnlp+machine+translation+systems+for+wmt24+low%2Dresource+indic+language+translation%2E+In+Proceedings+of+the+Low%2DResource+Indic+Language+T)
4. Vanlalmuansangi Khenglawt, Sahinur Rahman Laskar, Santanu Pal, Partha Pakray, and Ajoy Kumar Khan. 2022. Language resource building and English-to-mizo neural machine translation encountering tonal words. In Proceedings of the WILDRE-6 Workshop within the 13th Language Resources and Evaluation Confe… [find](https://scholar.google.com/scholar?q=Vanlalmuansangi+Khenglawt%2C+Sahinur+Rahman+Laskar%2C+Santanu+Pal%2C+Partha+Pakray%2C+and+Ajoy+Kumar+Khan%2E+2022%2E+Language+resource+building+and+English%2Dto%2Dmizo+neural+machine+translation+encountering+tonal+wo)
5. Lenin Laitonjam and Sanasam Ranbir Singh. 2021. Manipuri-English machine translation using comparable corpus. In Proceedings of the 4th Workshop on Technologies for MT of Low Resource Languages (LoResMT2021), pages 78–88, Virtual. Association for Machine Translation in the Americas. [find](https://scholar.google.com/scholar?q=Lenin+Laitonjam+and+Sanasam+Ranbir+Singh%2E+2021%2E+Manipuri%2DEnglish+machine+translation+using+comparable+corpus%2E+In+Proceedings+of+the+4th+Workshop+on+Technologies+for+MT+of+Low+Resource+Languages+%28LoRes)
6. Sahinur Rahman Laskar, Abdullah Faiz Ur Rahman Khilji Darsh Kaushik, Partha Pakray, and Sivaji Bandyopadhyay. 2021. EnKhCorp1.0: An English– Khasi corpus. In Proceedings of the 4th Workshop on Technologies for MT of Low Resource Languages (LoResMT2021), pages 89–95, Virtual. Association for Machine… [find](https://scholar.google.com/scholar?q=Sahinur+Rahman+Laskar%2C+Abdullah+Faiz+Ur+Rahman+Khilji+Darsh+Kaushik%2C+Partha+Pakray%2C+and+Sivaji+Bandyopadhyay%2E+2021%2E+EnKhCorp1%2E0%3A+An+English%E2%80%93+Khasi+corpus%2E+In+Proceedings+of+the+4th+Workshop+on+Technol)
7. Yash Bhaskar and Parameswari Krishnamurthy. 2024. Yes-mt’s submission to the low-resource indic language translation shared task in wmt 2024. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translation (WMT24), EMNLP, November 15-16, 2024, Miami,… [find](https://scholar.google.com/scholar?q=Yash+Bhaskar+and+Parameswari+Krishnamurthy%2E+2024%2E+Yes%2Dmt%E2%80%99s+submission+to+the+low%2Dresource+indic+language+translation+shared+task+in+wmt+2024%2E+In+Proceedings+of+the+Low%2DResource+Indic+Language+Translat)
8. Sahinur Rahman Laskar, Abdullah Faiz Ur Rahman Khilji, Partha Pakray, and Sivaji Bandyopadhyay. 2020. EnAsCorp1.0: English-Assamese corpus. In Proceedings of the 3rd Workshop on Technologies for MT of Low Resource Languages, pages 62–68, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sahinur+Rahman+Laskar%2C+Abdullah+Faiz+Ur+Rahman+Khilji%2C+Partha+Pakray%2C+and+Sivaji+Bandyopadhyay%2E+2020%2E+EnAsCorp1%2E0%3A+English%2DAssamese+corpus%2E+In+Proceedings+of+the+3rd+Workshop+on+Technologies+for+MT+of)
9. Neha Gajakos, Prashanth Nayak, Rejwanul Haque, and Andy Way. 2024. Adapt-mt submissions to wmt 2024 low-resource indic language translation task. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translation (WMT24), EMNLP, November 15-16, 2024, Mia… [find](https://scholar.google.com/scholar?q=Neha+Gajakos%2C+Prashanth+Nayak%2C+Rejwanul+Haque%2C+and+Andy+Way%2E+2024%2E+Adapt%2Dmt+submissions+to+wmt+2024+low%2Dresource+indic+language+translation+task%2E+In+Proceedings+of+the+Low%2DResource+Indic+Language+Tran)
10. Sahinur Rahman Laskar, Riyanka Manna, Partha Pakray, and Sivaji Bandyopadhyay. 2022. A domain specific parallel corpus and enhanced english-assamese neural machine translation. Computación y Sistemas, 26(4):1669–1687. [find](https://scholar.google.com/scholar?q=Sahinur+Rahman+Laskar%2C+Riyanka+Manna%2C+Partha+Pakray%2C+and+Sivaji+Bandyopadhyay%2E+2022%2E+A+domain+specific+parallel+corpus+and+enhanced+english%2Dassamese+neural+machine+translation%2E+Computaci%C3%B3n+y+Sistemas%2C)
11. Loitongbam Gyanendro Singh, Lenin Laitonjam, and Sanasam Ranbir Singh. 2016. Automatic syllabification for Manipuri language. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 349–357, Osaka, Japan. [find](https://scholar.google.com/scholar?q=Loitongbam+Gyanendro+Singh%2C+Lenin+Laitonjam%2C+and+Sanasam+Ranbir+Singh%2E+2016%2E+Automatic+syllabification+for+Manipuri+language%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conference+on+Comput)
12. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
13. Barry Haddow and Faheem Kirefu. 2020. Pmindia – a collection of parallel corpora of languages of india. [find](https://scholar.google.com/scholar?q=Barry+Haddow+and+Faheem+Kirefu%2E+2020%2E+Pmindia+%E2%80%93+a+collection+of+parallel+corpora+of+languages+of+india%2E)
14. Hideki Isozaki, Tsutomu Hirao, Kevin Duh, Katsuhito Sudoh, and Hajime Tsukada. 2010. Automatic evaluation of translation quality for distant language pairs. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 944– 952, Cambridge, MA. Association for Compu… [find](https://scholar.google.com/scholar?q=Hideki+Isozaki%2C+Tsutomu+Hirao%2C+Kevin+Duh%2C+Katsuhito+Sudoh%2C+and+Hajime+Tsukada%2E+2010%2E+Automatic+evaluation+of+translation+quality+for+distant+language+pairs%2E+In+Proceedings+of+the+2010+Conference+on+Em) [our card](https://md.research.elamur.ai/free/papers/isozaki-2010-automatic-evaluation-translation-quality-distant/)
15. Abhinav P M, Ketaki Shetye, and Parameswari Krishnamurthy. 2024. Mtnlp-iiith: Machine translation for low-resource indic languages. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translation (WMT24), EMNLP, November 15-16, 2024, Miami, Florida, U… [find](https://scholar.google.com/scholar?q=Abhinav+P+M%2C+Ketaki+Shetye%2C+and+Parameswari+Krishnamurthy%2E+2024%2E+Mtnlp%2Diiith%3A+Machine+translation+for+low%2Dresource+indic+languages%2E+In+Proceedings+of+the+Low%2DResource+Indic+Language+Translation+Shared)
16. Santanu Pal, Partha Pakray, Sahinur Rahman Laskar, Lenin Laitonjam, Vanlalmuansangi Khenglawt, Sunita Warjri, Pankaj Kundan Dadure, and Sandeep Kumar Dash. 2023. Findings of the WMT 2023 shared task on low-resource Indic language translation. In Proceedings of the Eighth Conference on Machine Transl… [find](https://scholar.google.com/scholar?q=Santanu+Pal%2C+Partha+Pakray%2C+Sahinur+Rahman+Laskar%2C+Lenin+Laitonjam%2C+Vanlalmuansangi+Khenglawt%2C+Sunita+Warjri%2C+Pankaj+Kundan+Dadure%2C+and+Sandeep+Kumar+Dash%2E+2023%2E+Findings+of+the+WMT+2023+shared+task+o) [our card](https://md.research.elamur.ai/free/papers/pal-2023-findings-wmt-2023-shared-task/)
17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
18. Pranamya Patil, Raghavendra HR, Aditya Raghuwanshi, and Kushal Verma. 2024. Srib-nmt’s submission to the indic mt shared task in wmt 2024. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translation (WMT24), EMNLP, November 15-16, 2024, Miami, Flo… [find](https://scholar.google.com/scholar?q=Pranamya+Patil%2C+Raghavendra+HR%2C+Aditya+Raghuwanshi%2C+and+Kushal+Verma%2E+2024%2E+Srib%2Dnmt%E2%80%99s+submission+to+the+indic+mt+shared+task+in+wmt+2024%2E+In+Proceedings+of+the+Low%2DResource+Indic+Language+Translation)
19. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Maja+Popovic%CC%81%2E+2015%2E+chrF%3A+character+n%2Dgram+F%2Dscore+for+automatic+MT+evaluation%2E+In+Proceedings+of+the+Tenth+Workshop+on+Statistical+Machine+Translation%2C+pages+392%E2%80%93395%2C+Lisbon%2C+Portugal%2E+Association+f) [our card](https://md.research.elamur.ai/free/papers/popovic-2015-chrf-character-n-gram-f/)
20. Bin Wei, Jiawei Zhen, Zongyao Li, Zhanglin Wu, Jiaxin Guo, Daimeng Wei, Zhiqiang Rao, Shaojun Li, Yuanchang Luo, Hengchao Shang, Jinlong Yang, Yuhao Xie, and Hao Yang. 2024. Machine translation advancements of low-resource indian languages by transfer learning. In Proceedings of the Low-Resource Ind… [find](https://scholar.google.com/scholar?q=Bin+Wei%2C+Jiawei+Zhen%2C+Zongyao+Li%2C+Zhanglin+Wu%2C+Jiaxin+Guo%2C+Daimeng+Wei%2C+Zhiqiang+Rao%2C+Shaojun+Li%2C+Yuanchang+Luo%2C+Hengchao+Shang%2C+Jinlong+Yang%2C+Yuhao+Xie%2C+and+Hao+Yang%2E+2024%2E+Machine+translation+advanc)
21. Saumitra Yadav, Ananya Mukherjee, and Manish Shrivastava. 2024. A3-108 controlling token generation in low resource machine translation systems. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translation (WMT24), EMNLP, November 15-16, 2024, Miam… [find](https://scholar.google.com/scholar?q=Saumitra+Yadav%2C+Ananya+Mukherjee%2C+and+Manish+Shrivastava%2E+2024%2E+A3%2D108+controlling+token+generation+in+low+resource+machine+translation+systems%2E+In+Proceedings+of+the+Low%2DResource+Indic+Language+Trans)
22. Matthew Theodore Roquea, Carlos Rafael Catalana, Dan John Velascoa, Manuel Antonio Rufinoa, and Jan Christian Blaise Cruza. 2024. Samsung r&d institute philippines @ wmt 2024 indic mt task. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translati… [find](https://scholar.google.com/scholar?q=Matthew+Theodore+Roquea%2C+Carlos+Rafael+Catalana%2C+Dan+John+Velascoa%2C+Manuel+Antonio+Rufinoa%2C+and+Jan+Christian+Blaise+Cruza%2E+2024%2E+Samsung+r%26d+institute+philippines+%40+wmt+2024+indic+mt+task%2E+In+Proceed)
23. Pramit Sahoo, Maharaj Brahma, and Maunendra Sankar Desarkar. 2024. Nlip-lab-iith low-resource mt system for wmt24 indic mt shared task. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translation (WMT24), EMNLP, November 15-16, 2024, Miami, Florid… [find](https://scholar.google.com/scholar?q=Pramit+Sahoo%2C+Maharaj+Brahma%2C+and+Maunendra+Sankar+Desarkar%2E+2024%2E+Nlip%2Dlab%2Diith+low%2Dresource+mt+system+for+wmt24+indic+mt+shared+task%2E+In+Proceedings+of+the+Low%2DResource+Indic+Language+Translation+Sh)
24. Hamees Sayed, Advait Joglekar, and Srinivasan Umesh. 2024. Spring lab iitm’s submission to low resource indic language translation shared task. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translation (WMT24), EMNLP, November 15-16, 2024, Miami… [find](https://scholar.google.com/scholar?q=Hamees+Sayed%2C+Advait+Joglekar%2C+and+Srinivasan+Umesh%2E+2024%2E+Spring+lab+iitm%E2%80%99s+submission+to+low+resource+indic+language+translation+shared+task%2E+In+Proceedings+of+the+Low%2DResource+Indic+Language+Transl)
25. Loitongbam Gyanendro Singh and Sanasam Ranbir Singh. 2017. Word polarity detection using syllable features for manipuri language. In 2017 International Conference on Asian Language Processing (IALP), pages 206–209. [find](https://scholar.google.com/scholar?q=Loitongbam+Gyanendro+Singh+and+Sanasam+Ranbir+Singh%2E+2017%2E+Word+polarity+detection+using+syllable+features+for+manipuri+language%2E+In+2017+International+Conference+on+Asian+Language+Processing+%28IALP%29%2C+)
26. Matthew Snover, Bonnie Dorr, Rich Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pages 223–231, Cambridge, Massachu… [find](https://scholar.google.com/scholar?q=Matthew+Snover%2C+Bonnie+Dorr%2C+Rich+Schwartz%2C+Linnea+Micciulla%2C+and+John+Makhoul%2E+2006%2E+A+study+of+translation+edit+rate+with+targeted+human+annotation%2E+In+Proceedings+of+the+7th+Conference+of+the+Assoc) [our card](https://md.research.elamur.ai/free/papers/snover-2006-study-translation-edit-rate-targeted/)
27. Noel Murphy Sören Dréano, Derek Molloy. 2024. Exploration of the cyclegn framework for low-resource languages. In Proceedings of the Low-Resource Indic Language Translation Shared Task, 9th Conference on Machine Translation (WMT24), EMNLP, November 15-16, 2024, Miami, Florida, USA. [find](https://scholar.google.com/scholar?q=Noel+Murphy+S%C3%B6ren+Dr%C3%A9ano%2C+Derek+Molloy%2E+2024%2E+Exploration+of+the+cyclegn+framework+for+low%2Dresource+languages%2E+In+Proceedings+of+the+Low%2DResource+Indic+Language+Translation+Shared+Task%2C+9th+Conference)