---
title: "Hala Technical Report Building Arabic-Centric Instruction &"
description: "Arabic language models are held back by a shortage of good Arabic instruction-tuning data. 48.2 BLEU. 51 references, cited by 5 papers here."
authors: ["Hasan Abed Al Kader Hammoud", "Mohamad Bilal Zbib", "Bernard Ghanem"]
year: 2026
venue: "Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script"
source: "https://aclanthology.org/2026.abjadnlp-1.32/"
doi: "10.18653/v1/2026.abjadnlp-1.32"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hammoud-2026-hala-technical-report-building-arabic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hammoud-2026-hala-technical-report-building-arabic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/) / Hala Technical Report Building Arabic-Centric Instruction & Translation Models at Scale

# Hala Technical Report Building Arabic-Centric Instruction & Translation Models at Scale

**Authors:** [Hasan Abed Al Kader Hammoud](https://md.research.elamur.ai/free/en/authors/hasan-abed-al-kader-hammoud/), [Mohamad Bilal Zbib](https://md.research.elamur.ai/free/en/authors/mohamad-bilal-zbib/) and [Bernard Ghanem](https://md.research.elamur.ai/free/en/authors/bernard-ghanem/) · **Year:** 2026 · **Venue:** [Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script](https://md.research.elamur.ai/free/en/venues/abjadnlp/) ([Workshop on NLP for Languages Using Arabic Script 2026](https://md.research.elamur.ai/free/en/venues/abjadnlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.abjadnlp-1.32/) · [PDF](https://aclanthology.org/2026.abjadnlp-1.32.pdf) · [DOI 10.18653/v1/2026.abjadnlp-1.32](https://doi.org/10.18653/v1/2026.abjadnlp-1.32) · [OpenAlex](https://openalex.org/W7140094238)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [5](https://md.research.elamur.ai/free/en/papers/hammoud-2026-hala-technical-report-building-arabic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic language models are held back by a shortage of good Arabic instruction-tuning data. The authors address this with a "translate-and-tune" pipeline. First, a large AR↔EN translator, CohereLabs/command-a-translate-08-2025, was quantized to FP8. It ran about twice as fast and its translation scores did not drop. This teacher translated part of Open-Orca, and its output was combined with a filtered portion of OPUS-100. The resulting bilingual set was used to fine-tune the small LiquidAI/LFM2-1.2B model into a fast translator. That small translator then converted several English instruction collections into Arabic, giving roughly 4.5M samples.

On this data they fine-tuned models at 350M, 700M, 1.2B (LFM2 bases) and 9B (Fanar-1-9B-Instruct base). Each fine-tuned model was then slerp-merged back with its base. The HALA models achieve the best average scores in the ≤2B and 7–9B classes on an OALL-style Arabic benchmark suite. The work matters because it shows a cheap, reproducible route to depth in a single language, and all models and data are released openly.

## Key points
- **Bilingual training set (~1.26M examples):**
  - 405K Open-Orca instruction–response pairs translated by the FP8 teacher, counted in both directions to give 810K examples.
  - 439,592 OPUS-100 ar-en pairs kept out of about 1M. A Qwen2.5-3B-Instruct judge gave each pair a binary accept/reject verdict.
- **Arabic instruction corpus (~4.5M samples):** translated by the small translator from Open-Orca, Hermes 3, SCP-116K, ReAlign-Alpaca, LaMini, Tulu 3 (English subsets only) and Synthetic Instruct-GPT-J Pairwise.
- **Translation quality** was measured on 500 sampled MMLU questions, using the matching items in openai/mmmlu Arabic as references:
  - FP8 and FP16 teachers score about the same (BLEU ≈53, chrF++ ≈69).
  - The fine-tuned LFM2-1.2B translator rises from 16.0 to 48.2 BLEU and from 43.2 to 64.2 chrF++.
- **Benchmark averages, base → HALA:**
  - 350M: 34.3 → 41.9
  - 700M: 41.4 → 46.9
  - 1.2B: 46.3 → 51.4
  - 9B: 69.2 → 69.9, ahead of Fanar-1-9B-Instruct
- **Merging** used MergeKit slerp at t=0.5. Benchmarks included AlGhafa, AraTrust and ArabicMMLU, among others.

## Interesting ideas
- A small instruction-tuned model, given a strict yes/no prompt, can act as a cheap filter that discards weak pairs from a large noisy parallel corpus; this scales better than human checking.
- Knowledge is passed down in steps: a big teacher is made cheaper through quantization, its output trains a tiny specialist, and that specialist then does the bulk of the work, so the expensive model touches only a seed set.
- After narrow fine-tuning, interpolating the weights back toward the base model (slerp at the midpoint) keeps general ability while retaining most of the specialization.
- Translation fidelity can be scored without new human references by reusing a benchmark that already exists in aligned versions per language, matched by item ID.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): A compact LLM could give accept/reject verdicts on silver UD trees or frames where the witness parsers disagree, cleaning the training pool before it is used.
- **I2** (learn): A costly LLM teacher tier could label a seed set to train a small fast model, which then annotates the large corpus, cutting cost while keeping teacher-level quality.
- **I3** (tree): A gparse model fine-tuned on one treebank or UD dialect could be merged with the general checkpoint to avoid forgetting other domains.
- **I4** (back): The text → graph → text and meta-world translation paths could be scored against parallel benchmark items such as MMLU/MMMLU with chrF++ or BLEU, giving a reference-based check of generation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [King Abdullah University of Science and Technology](https://md.research.elamur.ai/free/en/universities/king-abdullah-university-of-science-and-technology/) — University, Jeddah, Saudi Arabia · 27 works · 1,084 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 1,087 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Hasan Abed Al Kader Hammoud, Mohamad Bilal Zbib and Bernard Ghanem. 2026. Hala Technical Report Building Arabic-Centric Instruction & Translation Models at Scale. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hammoud2026hala,
  title = {{Hala Technical Report Building Arabic-Centric Instruction \& Translation Models at Scale}},
  author = {Hasan Abed {Al Kader Hammoud}, Mohamad Bilal Zbib and Bernard Ghanem},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.32},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.32}
}
```

- [← previous GLARE: Agentic Reasoning for Legal Judgment Prediction](https://md.research.elamur.ai/free/en/papers/yang-2026-glare-agentic-reasoning-legal-judgment/)
- [next → HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance](https://md.research.elamur.ai/free/en/papers/zhang-2026-hyperadalora-accelerating-lora-rank-allocation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/)

## References

51 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774. [↗](https://arxiv.org/abs/2303.08774)
2. Red Hat AI and vLLM Project. 2024. LLM Compressor. [find](https://scholar.google.com/scholar?q=Red+Hat+AI+and+vLLM+Project%2E+2024%2E+LLM+Compressor%2E)
3. Shahad Al-Khalifa, Nadir Durrani, Hend Al-Khalifa, and Firoj Alam. 2025. [The landscape of Arabic large language models (ALLMs): A new era for Arabic language technology](https://md.research.elamur.ai/free/en/papers/al-khalifa-2025-landscape-arabic-large-language-models/). Preprint, arXiv:2506.01340. [↗](https://arxiv.org/abs/2506.01340)
4. Emad A Alghamdi, Reem I Masoud, Deema Alnuhait, Afnan Y Alomairi, Ahmed Ashraf, and Mohamed Zaytoon. 2024. [Aratrust: An evaluation of trustworthiness for llms in arabic](https://md.research.elamur.ai/free/en/papers/alghamdi-2025-aratrust-evaluation-trustworthiness-llms-arabic/). arXiv preprint arXiv:2403.09017. [↗](https://arxiv.org/abs/2403.09017)
5. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023a. [The falcon series of open language models](https://md.research.elamur.ai/free/en/papers/almazrouei-2023-falcon-series-open-language-models/). Preprint, arXiv:2311.16867. [↗](https://arxiv.org/abs/2311.16867)
6. Ebtesam Almazrouei, Ruxandra Cojocaru, Michele Baldo, Quentin Malartic, Hamza Alobeidli, Daniele Mazzotta, Guilherme Penedo, Giulia Campesan, Mugariya Farooq, Maitha Alhammadi, Julien Launay, and Badreddine Noune. 2023b. [AlGhafa evaluation benchmark for Arabic language models](https://md.research.elamur.ai/free/en/papers/almazrouei-2023-alghafa-evaluation-benchmark-arabic-language/). In Proceedings of ArabicNLP 2023, pages 244–275, Singapore (Hybrid). Association for Computational Linguistics.
7. Fakhraddin Alwajih, El Moatez Billah Nagoudi, Gagan Bhatia, Abdelrahman Mohamed, and Muhammad Abdul-Mageed. 2024. [Peacock: A family of Arabic multimodal large language models and benchmarks](https://md.research.elamur.ai/free/en/papers/alwajih-2024-peacock-family-arabic-multimodal-large/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12753–12776, Bangkok, Thailand. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2403.01031)
8. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
9. M. Saiful Bari, Yazeed Alnumay, Norah A. Alzahrani, Nouf M. Alotaibi, Hisham A. Alyahya, Sultan Al-Rashed, Faisal A. Mirza, Shaykhah Z. Alsubaie, Hassan A. Alahmed, Ghadah Alabduljabbar, Raghad Alkhathran, Yousef Almushayqih, Raneem Alnajim, Maryam Al Mansour, Majed Alrubaian, Ali Alammari, Zaki Alawami, Abdulmohsen Al-Thubaity, Ahmed Abdelali, and 5 others. 2024. [ALLaM: Large language models for Arabic and english](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/). Preprint, arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390)
10. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc.
11. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.747)
12. Ona de Gibert, Graeme Nail, Nikolay Arefyev, Marta Bañón, Jelmer van der Linde, Shaoxiong Ji, Jaume Zaragoza-Bernabeu, Mikko Aulamo, Gema Ramírez-Sánchez, Andrey Kutuzov, Sampo Pyysalo, Stephan Oepen, and Jörg Tiedemann. 2024. [A new massive multilingual dataset for high-performance language technologies](https://md.research.elamur.ai/free/en/papers/gibert-2024-new-massive-multilingual-dataset-high/). In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1116–1128, Torino, Italia. ELRA and ICCL.
13. Run-Ze Fan, Xuefeng Li, Haoyang Zou, Junlong Li, Shwai He, Ethan Chern, Jiewen Hu, and Pengfei Liu. 2024. Reformatted alignment. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 574–597, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Run%2DZe+Fan%2C+Xuefeng+Li%2C+Haoyang+Zou%2C+Junlong+Li%2C+Shwai+He%2C+Ethan+Chern%2C+Jiewen+Hu%2C+and+Pengfei+Liu%2E+2024%2E+Reformatted+alignment%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+202)
14. Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuwis, Gokhan Tur, and Prem Natarajan. 2023. MASSIVE: A 1M-example multilingual natural language understanding dataset with 51 typologically-diverse languages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4277–4302, Toronto, Canada. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2204.08582)
15. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Jean-bastien Grill, Geoffrey Cideron, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, and 14 others. 2025. Gemma 3 technical report. Preprint, arXiv:2503.19786. Author list abbreviated. [↗](https://arxiv.org/abs/2503.19786)
16. Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. [Arcee’s MergeKit: A toolkit for merging large language models](https://md.research.elamur.ai/free/en/papers/goddard-2024-arcee-s-mergekit-toolkit-merging/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 477–485, Miami, Florida, US. Association for Computational Linguistics. [↗](https://doi.org/10.13140/rg.2.2.10481.70249)
17. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, ..., Xiaolan Ma, Xilun Wang, and Yossi Adi. 2024. [The Llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). Preprint, arXiv:2407.21783. Author list abbreviated. [↗](https://arxiv.org/abs/2407.21783)
18. Nathan Habib, Clémentine Fourrier, Hynek Kydlíček, Thomas Wolf, and Lewis Tunstall. 2023. Lighteval: A lightweight framework for llm evaluation. serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles. [find](https://scholar.google.com/scholar?q=Nathan+Habib%2C+Cl%C3%A9mentine+Fourrier%2C+Hynek+Kydl%C3%ADc%CC%8Cek%2C+Thomas+Wolf%2C+and+Lewis+Tunstall%2E+2023%2E+Lighteval%3A+A+lightweight+framework+for+llm+evaluation%2E+serving+with+pagedattention%2E+In+Proceedings+of+the+ACM)
19. Momchil Hardalov, Todor Mihaylov, Dimitrina Zlatkova, Yoan Dinkov, Ivan Koychev, and Preslav Nakov. 2020. EXAMS: A multi-subject high school examinations dataset for cross-lingual and multilingual question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5427–5444, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Momchil+Hardalov%2C+Todor+Mihaylov%2C+Dimitrina+Zlatkova%2C+Yoan+Dinkov%2C+Ivan+Koychev%2C+and+Preslav+Nakov%2E+2020%2E+EXAMS%3A+A+multi%2Dsubject+high+school+examinations+dataset+for+cross%2Dlingual+and+multilingual+que)
20. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, and 4 others. 2024. [Tülu 3: Pushing frontiers in open language model post-training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/). https://arxiv.org/abs/2411.15124. [↗](https://arxiv.org/abs/2411.15124)
21. Alex Havrilla. 2023. synthetic-instruct-gptj-pairwise (revision cc92d8d). [find](https://scholar.google.com/scholar?q=Alex+Havrilla%2E+2023%2E+synthetic%2Dinstruct%2Dgptj%2Dpairwise+%28revision+cc92d8d%29%2E)
22. Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, and Xing Wang. 2024. [Exploring human-like translation strategy with large language models](https://md.research.elamur.ai/free/en/papers/he-2024-exploring-human-like-translation-strategy/). Transactions of the Association for Computational Linguistics, 12:229–246. [↗](https://doi.org/10.1162/tacl_a_00642)
23. Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, and 2 others. 2022. [Few-shot learning with multilingual language models](https://md.research.elamur.ai/free/en/papers/lin-2021-few-shot-learning-multilingual-language/). Preprint, arXiv:2112.10668. [↗](https://arxiv.org/abs/2112.10668)
24. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Song Dingjie, Zhihong Chen, Mosen Alharthi, Bang An, Juncai He, Ziche Liu, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, and Jinchao Xu. 2024. [AceGPT, localizing large language models in Arabic](https://md.research.elamur.ai/free/en/papers/huang-2024-acegpt-localizing-large-language-models/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8139–8163, Mexico City, Mexico. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2309.12053)
25. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. [↗](https://arxiv.org/abs/2412.19437)
26. Maxim Ifergan, Leshem Choshen, Roee Aharoni, Idan Szpektor, and Omri Abend. 2024. Beneath the surface of consistency: Exploring cross-lingual knowledge representation sharing in LLMs. Preprint, arXiv:2408.10646. [↗](https://arxiv.org/abs/2408.10646)
27. Chenyang Lyu, Zefeng Du, Jitao Xu, Yitao Duan, Minghao Wu, Teresa Lynn, Alham Fikri Aji, Derek F. Wong, and Longyue Wang. 2024. A paradigm shift: The future of machine translation lies with large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1339–1352, Torino, Italia. ELRA and ICCL. [find](https://scholar.google.com/scholar?q=Chenyang+Lyu%2C+Zefeng+Du%2C+Jitao+Xu%2C+Yitao+Duan%2C+Minghao+Wu%2C+Teresa+Lynn%2C+Alham+Fikri+Aji%2C+Derek+F%2E+Wong%2C+and+Longyue+Wang%2E+2024%2E+A+paradigm+shift%3A+The+future+of+machine+translation+lies+with+large+lang)
28. Hannah Calzi Kleidermacher and James Zou. 2025. Science across languages: Assessing LLM multilingual translation of scientific papers. arXiv preprint arXiv:2502.17882. [↗](https://arxiv.org/abs/2502.17882)
29. "Fajri Koto, Haonan Li, Sara Shatanawi, Jad Doughman, Abdelrahman Boda Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, and Timothy Baldwin". 2024. [Arabicmmlu: Assessing massive multitask language understanding in arabic](https://md.research.elamur.ai/free/en/papers/koto-2024-arabicmmlu-assessing-massive-multitask-language/). In Findings of the Association for Computational Linguistics: ACL 2024.
30. Andrey Kuzmin, Mart Van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, and Tijmen Blankevoort. 2022. Fp8 quantization: The power of the exponent. Advances in Neural Information Processing Systems, 35:14651–14662. [find](https://scholar.google.com/scholar?q=Andrey+Kuzmin%2C+Mart+Van+Baalen%2C+Yuwei+Ren%2C+Markus+Nagel%2C+Jorn+Peters%2C+and+Tijmen+Blankevoort%2E+2022%2E+Fp8+quantization%3A+The+power+of+the+exponent%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A1)
31. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model Dakuan Lu, Xiaoyu Tan, Rui Xu, Tianchu Yao, Chao Qu, Wei Chu, Yinghui Xu, and Yuan Qi. 2025. Scp-116k: A high-quality problem-solution dataset and a generalized pipeline for automated extraction in the higher education science domain. arXiv preprint arXiv:2501.15587. [↗](https://arxiv.org/abs/2501.15587)
32. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. [Orca: Progressive learning from complex explanation traces of GPT–4](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/). Preprint, arXiv:2306.02707. [↗](https://arxiv.org/abs/2306.02707)
33. Navid-AI. 2025. Yehia 7b preview. [https://huggingface.co/Navid-AI/Yehia-7B-preview](https://huggingface.co/Navid-AI/Yehia-7B-preview). [↗](https://huggingface.co/Navid-AI/Yehia-7B-preview)
34. Sunny Sanyal, Atula Neerkaje, Jean Kaddour, Abhishek Kumar, and Sujay Sanghavi. 2023. Early weight averaging meets high learning rates for llm pre-training. Preprint, arXiv:2306.03241. [↗](https://arxiv.org/abs/2306.03241)
35. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, and 9 others. 2023. [Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models](https://md.research.elamur.ai/free/en/papers/sengupta-2023-jais-jais-chat-arabic-centric/). Preprint, arXiv:2308.16149. silma-ai. 2024. Silma 9b instruct https://huggingface.co/silma-ai/ SILMA-9B-Instruct-v1.0. v1.0. [↗](https://arxiv.org/abs/2308.16149)
36. SILMA-AI. 2025. Silma kashif 2b instruct v1.0. [https://huggingface.co/silma-ai/](https://huggingface.co/silma-ai/) SILMA-Kashif-2B-Instruct-v1.0. [↗](https://huggingface.co/silma-ai/)
37. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, Majd Hawasly, and 22 others. 2025a. [Fanar: An Arabic-centric multimodal generative AI platform](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/). Preprint, arXiv:2501.13944. [↗](https://arxiv.org/abs/2501.13944)
38. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805. [↗](https://arxiv.org/abs/2312.11805)
39. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others. 2025b. [Kimi k2: Open agentic intelligence](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k2-open-agentic/). arXiv preprint arXiv:2507.20534. [↗](https://arxiv.org/abs/2507.20534)
40. Ryan Teknium, Jeffrey Quesnelle, and Guang Chen. 2024. Hermes 3 technical report. Preprint, arXiv:2408.11857. [↗](https://arxiv.org/abs/2408.11857)
41. Jörg Tiedemann. 2020. [The tatoeba translation challenge – realistic data sets for low resource and multilingual MT](https://md.research.elamur.ai/free/en/papers/tiedemann-2020-tatoeba-translation-challenge-realistic-data/). In Proceedings of the Fifth Conference on Machine Translation, pages 1174–1182, Online. Association for Computational Linguistics.
42. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). Preprint, arXiv:2307.09288. [↗](https://arxiv.org/abs/2307.09288)
43. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13484–13508, Toronto, Canada. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2212.10560)
44. BigScience Workshop, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, and 1 others. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. [↗](https://arxiv.org/abs/2211.05100)
45. Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, and Alham Fikri Aji. 2024. LaMini-LM: A diverse herd of distilled models from large-scale instructions. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 944–964, St. Julian’s, Malta. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2304.14402)
46. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). arXiv preprint arXiv:2304.12244. [↗](https://arxiv.org/abs/2304.12244)
47. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. [mT5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.naacl-main.41)
48. Ai Ming Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, Fei Deng, Feng Wang, Feng Liu, Guangwei Ai, Guosheng Dong, Hai Zhao, Hang Xu, Hao-Lun Sun, and 36 others. 2023. [Baichuan 2: Open large-scale language models](https://md.research.elamur.ai/free/en/papers/yang-2023-baichuan-2-open-large-scale/). ArXiv, abs/2309.10305. [↗](https://arxiv.org/abs/2309.10305)
49. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388. [↗](https://arxiv.org/abs/2505.09388)
50. Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. [Improving massively multilingual neural machine translation and zero-shot translation](https://md.research.elamur.ai/free/en/papers/zhang-2020-improving-massively-multilingual-neural-machine/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1628– 1639, Online. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2004.11867)
51. Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2024. [Multilingual machine translation with large language models: Empirical results and analysis](https://md.research.elamur.ai/free/en/papers/zhu-2024-multilingual-machine-translation-large-language/). In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2765–2781, Mexico City, Mexico. Association for Computational Linguistics.

*Updated 1 November 2026*