---
title: "SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic — Alrashed 2025"
description: "The authors build SmolKalam, an Arabic version of a curated subset of SmolTalk2. 28 references."
authors: ["Sultan Alrashed", "Chadi Helwe", "Francesco Orabona"]
year: 2025
venue: "arXiv:2511.18411"
source: "https://arxiv.org/abs/2511.18411"
doi: "10.48550/arxiv.2511.18411"
arxiv: "2511.18411"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/alrashed-2025-smolkalam-ensemble-quality-filtered-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alrashed-2025-smolkalam-ensemble-quality-filtered-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/) / SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Data

# SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Data

**Authors:** [Sultan Alrashed](https://md.research.elamur.ai/free/en/authors/sultan-alrashed/), [Chadi Helwe](https://md.research.elamur.ai/free/en/authors/chadi-helwe/) and Francesco Orabona · **Year:** 2025 · **Venue:** arXiv:2511.18411 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2511.18411) · [PDF](https://arxiv.org/pdf/2511.18411) · [DOI 10.48550/arxiv.2511.18411](https://doi.org/10.48550/arxiv.2511.18411) · [arXiv 2511.18411](https://arxiv.org/abs/2511.18411) · [OpenAlex](https://openalex.org/W7106682941)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Arabic LLMs have reasonable pre-training corpora, but large post-training sets are scarce. Such sets need multi-turn chats, reasoning traces and tool calls, and unfiltered machine translation is too noisy for this stage. The authors build SmolKalam, an Arabic version of a curated subset of SmolTalk2. Each sample is translated by two systems: Seed-X 7B running locally on V100 GPUs, and Gemma 3 accessed through OpenRouter. The candidates are then ranked with a small Qwen-based reward model and two cheap intrinsic scores. Language Ratio (LR) checks that the target length stays close to the source length. Script Purity (SCR) measures the share of Arabic-script characters once code, URLs and math are set aside. The released corpus has about 1.5–1.8M examples (2.8–3.3B tokens), depending on which translator is primary. Ablations show that chunk size and sampling temperature both affect translation quality. The resource matters because it brings multi-turn dialogue, tool use and `<think>` reasoning into Arabic instruction tuning at scale, and the full pipeline is released.

## Key points
- Two ranked configurations are released:
  - SFT_SeedX_ranked: 1,777,275 examples, about 3.26B tokens, mean LR 0.796, SCR 0.925.
  - SFT_Gemma3_ranked: 1,545,742 examples, about 2.82B tokens, mean LR 0.808, SCR 0.928.
- How the local pipeline works:
  - It splits text into roughly 490-token chunks to fit Seed-X's 512-token window.
  - Cuts are placed preferably at sentence boundaries within ±50 tokens of the target position.
  - Role, message index and chunk metadata are stored so each sample can be rebuilt exactly.
  - Translating the whole set took about 600 GPU-hours.
- How the reward model was trained:
  - The model is Qwen 2.5 1.5B with a Bradley-Terry objective.
  - The preferences come from many models' translations of S1K.
  - These translations were ranked by the Arabic MMLU scores of small Qwen3 4B and Llama 3.2 3B students fine-tuned on each one.
- Results on S1K translations:
  - GPT 4.1 ranked best; Gemma3 27B and Qwen3 32B came next.
  - Small models such as Qwen2.5 0.5B–3B and Falcon3 3B collapsed on LR (below 0.2).
  - The two student families sometimes ordered the translators very differently; Qwen3 32B was 1st for Llama but 7th for Qwen.
- Ablations with Qwen3 32B:
  - 25-line chunks beat larger chunks, and LR fell to 0.63 with 500-line chunks.
  - Temperature 0.7 scored slightly above greedy decoding.

## Interesting ideas
- Instead of trusting a generic quality score, rank candidate data sources by fine-tuning small student models on each and measuring a downstream benchmark, then distill that ranking into a cheap pairwise reward model.
- Produce several candidate outputs per item from different systems and choose the best one per sample, rather than picking a single winning system for the whole corpus.
- Use cheap label-free sanity scores: a soft exponential penalty on output/source length divergence taken as the minimum over word and character counts, plus a script-share check that excludes whitelisted spans and saturates at a 90% threshold.
- Split long inputs near natural sentence boundaries within a tolerance window, and attach position metadata so the pieces can be reassembled deterministically.
- Check whether quality rankings stay stable across different student architectures, since one backbone can reorder candidates sharply.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Each LLM teacher tier's silver annotations could be judged by training a small perceptron parser or frame labeler on them and scoring on gold, and a lightweight ranker could then be learned from those outcomes.
- **I2** (learn): When the witness parsers disagree, a learned per-sentence selector could choose which parse becomes silver data, instead of only keeping cases where they agree.
- **I3** (back): Texts generated from the meaning graph, or passed through the compressor round trip, could be flagged when their length drifts from the source, at almost zero cost.
- **I4** (tree): The gparse transformer parser could process long documents in this way within its context limit and still rebuild document-level token indices exactly.
- **I5** (answer): Comparisons of reader or student variants on school tests could be repeated with two different underlying models, so conclusions do not hinge on one backbone's quirks.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/)

## Cite

Cite as: Sultan Alrashed, Chadi Helwe and Francesco Orabona. 2025. SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Data. arXiv:2511.18411.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{alrashed2025smolkalam,
  title = {{SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Data}},
  author = {Sultan Alrashed, Chadi Helwe and Francesco Orabona},
  year = {2025},
  doi = {10.48550/arxiv.2511.18411},
  eprint = {2511.18411},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2511.18411}
}
```

- [← previous Sentence Alignment of Ancient and Modern Chinese Based on Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/yu-2025-sentence-alignment-ancient/)
- [next → Some Dynamics of Word Order in Biblical Hebrew](https://md.research.elamur.ai/free/en/papers/joosten-2025-some-dynamics-word-order-biblical/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/)

## References

28 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Zaid Alyafeai, Khalid Almubarak, Ahmed Ashraf, Deema Alnuhait, Saied Alshahrani, Gubran Abdulrahman, Gamil Ahmed, Qais Gawah, Zead Saleh, Mustafa Ghaleb, et al. 2024. Cidar: Culturally relevant instruction dataset for arabic. In Findings of the Association for Computational Linguistics: ACL 2024, pages 12878–12901. [find](https://scholar.google.com/scholar?q=Zaid+Alyafeai%2C+Khalid+Almubarak%2C+Ahmed+Ashraf%2C+Deema+Alnuhait%2C+Saied+Alshahrani%2C+Gubran+Abdulrahman%2C+Gamil+Ahmed%2C+Qais+Gawah%2C+Zead+Saleh%2C+Mustafa+Ghaleb%2C+et+al%2E+2024%2E+Cidar%3A+Culturally+relevant+instru)
2. Mikko Aulamo, Sami Virpioja, and Jörg Tiedemann. 2020. OpusFilter: A configurable parallel corpus filtering toolbox. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 150–156, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mikko+Aulamo%2C+Sami+Virpioja%2C+and+J%C3%B6rg+Tiedemann%2E+2020%2E+OpusFilter%3A+A+configurable+parallel+corpus+filtering+toolbox%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Ling)
3. Elie Bakouch, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Lewis Tunstall, Carlos Miguel Patiño, Edward Beeching, Aymeric Roucher, Aksel Joonas Reedi, Quentin Gallouédec, Kashif Rasul, Nathan Habib, Clémentine Fourrier, Hynek Kydlicek, Guilherme Penedo, Hugo Larcher, Mathieu Morlon, Vaibhav Srivastav, Joshua Lochner, Xuan-Son Nguyen, Colin Raffel, Leandro von Werra, and Thomas Wolf. 2025. SmolLM3: smol, multilingual, long-context reasoner. [https://huggingface.co/blog/smollm3](https://huggingface.co/blog/smollm3). [↗](https://huggingface.co/blog/smollm3)
4. Marta Bañón, Pinzhen Chen, Barry Haddow, Kenneth Heafield, Hieu Hoang, Miquel Esplà-Gomis, Mikel L. Forcada, Amir Kamran, Faheem Kirefu, Philipp Koehn, Sergio Ortiz Rojas, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Elsa Sarrías, Marek Strelec, Brian Thompson, William Waites, Dion Wiggins, and Jaume Zaragoza. 2020. ParaCrawl: Web-scale acquisition of parallel corpora. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4555–4567, Online. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/banon-2020-paracrawl-web-scale-acquisition-parallel/)
5. M Saiful Bari, Yazeed Alnumay, Norah A Alzahrani, Nouf M Alotaibi, Hisham A Alyahya, Sultan AlRashed, Faisal A Mirza, Shaykhah Z Alsubaie, Hassan A Alahmed, Ghadah Alabduljabbar, et al. 2024. Allam: Large language models for arabic and english. arXiv preprint arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390) [our card](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/)
6. Ralph Allan Bradley and Milton E. Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324–345. [find](https://scholar.google.com/scholar?q=Ralph+Allan+Bradley+and+Milton+E%2E+Terry%2E+1952%2E+Rank+analysis+of+incomplete+block+designs%3A+I%2E+the+method+of+paired+comparisons%2E+Biometrika%2C+39%283%2F4%29%3A324%E2%80%93345%2E)
7. Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. 2023a. Alpagasus: Training a better alpaca with fewer data. arXiv preprint arXiv:2307.08701. [↗](https://arxiv.org/abs/2307.08701) [our card](https://md.research.elamur.ai/free/en/papers/chen-2023-alpagasus-training-better-alpaca-fewer/)
8. Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, et al. 2023b. Phoenix: Democratizing chatgpt across languages. arXiv preprint arXiv:2304.10453. [↗](https://arxiv.org/abs/2304.10453)
9. Shanbo Cheng, Yu Bao, Qian Cao, Luyang Huang, Liyan Kang, Zhicheng Liu, Yu Lu, Wenhao Zhu, Jingwen Chen, Zhichao Huang, Tao Li, Yifu Li, Huiying Lin, Sitong Liu, Ningxin Peng, Shuaijie She, Lu Xu, Nuo Xu, Sen Yang, Runsheng Yu, Yiming Yu, Liehao Zou, Hang Li, Lu Lu, Yuxuan Wang, and Yonghui Wu. 2025. Seed-x: Building strong multilingual translation llm with 7b parameters. [find](https://scholar.google.com/scholar?q=Shanbo+Cheng%2C+Yu+Bao%2C+Qian+Cao%2C+Luyang+Huang%2C+Liyan+Kang%2C+Zhicheng+Liu%2C+Yu+Lu%2C+Wenhao+Zhu%2C+Jingwen+Chen%2C+Zhichao+Huang%2C+Tao+Li%2C+Yifu+Li%2C+Huiying+Lin%2C+Sitong+Liu%2C+Ningxin+Peng%2C+Shuaijie+She%2C+Lu+Xu%2C+Nuo)
10. William A. Gale and Kenneth W. Church. 1993. A program for aligning sentences in bilingual corpora. Computational Linguistics, 19(1):75–102. [our card](https://md.research.elamur.ai/free/en/papers/gale-1993-program-aligning-sentences-bilingual-corpora/)
11. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. 2020. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. 2024. The llama 3 herd of models. [↗](https://arxiv.org/abs/2101.00027)
12. Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shreyas Pimpalgaonkar, Kartik Sharma, Charlie Cheng-Jie Ji, Yichuan Deng, Sarah Pratt, Vivek Ramanujan, Jon Saad-Falcon, Jeffrey Li, Achal Dave, Alon Albalak, Kushal Arora, Blake Wulfe, Chinmay Hegde, Greg Durrett, Sewoong Oh, Mohit Bansal, Saadia Gabriel, Aditya Grover, Kai-Wei Chang, Vaishaal Shankar, Aaron Gokaslan, Mike A. Merrill, Tatsunori Hashimoto, Yejin Choi, Jenia Jitsev, Reinhard Heckel, Maheswaran Sathiamoorthy, Alexandros G. Dimakis, and Ludwig Schmidt. 2025. Openthoughts: Data recipes for reasoning models. [our card](https://md.research.elamur.ai/free/en/papers/guha-2025-openthoughts-data-recipes-reasoning-models/)
13. Hasan Abed Al Kader Hammoud, Mohammad Zbeeb, and Bernard Ghanem. 2025. Hala technical report: Building arabic-centric instruction & translation models at scale. arXiv preprint arXiv:2509.14008. [↗](https://arxiv.org/abs/2509.14008)
14. Junliang He, Ziyue Fan, Shaohui Kuang, Li Xiaoqing, Kai Song, Yaqian Zhou, and Xipeng Qiu. 2025. Fine: Filtering and improving noisy data elaborately with large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8686–8707. [find](https://scholar.google.com/scholar?q=Junliang+He%2C+Ziyue+Fan%2C+Shaohui+Kuang%2C+Li+Xiaoqing%2C+Kai+Song%2C+Yaqian+Zhou%2C+and+Xipeng+Qiu%2E+2025%2E+Fine%3A+Filtering+and+improving+noisy+data+elaborately+with+large+language+models%2E+In+Proceedings+of+the+)
15. Erik Henriksson, Otto Tarkka, and Filip Ginter. 2025. Finerweb-10bt: Refining web data with llm-based line-level filtering. arXiv preprint arXiv:2501.07314. [↗](https://arxiv.org/abs/2501.07314)
16. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Song Dingjie, Zhihong Chen, Mosen Alharthi, Bang An, Juncai He, et al. 2024. Acegpt, localizing large language models in arabic. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8139–8163. [our card](https://md.research.elamur.ai/free/en/papers/huang-2024-acegpt-localizing-large-language-models/)
17. Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, et al. 2024. Arabicmmlu: Assessing massive multitask language understanding in arabic. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5622–5640. [our card](https://md.research.elamur.ai/free/en/papers/koto-2024-arabicmmlu-assessing-massive-multitask-language/)
18. Viet Lai, Chien Nguyen, Nghia Ngo, Thuat Nguyen, Franck Dernoncourt, Ryan Rossi, and Thien Nguyen. 2023. Okapi: Instruction-tuned large language models in multiple languages with reinforcement learning from human feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 318–327. [our card](https://md.research.elamur.ai/free/en/papers/lai-2023-okapi-instruction-tuned-large-language/)
19. Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin. 2023. Bactrian-x: Multilingual replicable instruction-following models with low-rank adaptation. arXiv preprint arXiv:2305.15011. [↗](https://arxiv.org/abs/2305.15011)
20. Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. 2024. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning. arXiv preprint arXiv:2402.00530. [↗](https://arxiv.org/abs/2402.00530)
21. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. [our card](https://md.research.elamur.ai/free/en/papers/muennighoff-2025-s1-simple-test-time-scaling/)
22. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707. [↗](https://arxiv.org/abs/2306.02707) [our card](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/)
23. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67. [our card](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/)
24. Lucia Specia, Kashif Shah, Jose G.C. de Souza, and Trevor Cohn. 2013. QuEst - a translation quality estimation framework. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 79–84, Sofia, Bulgaria. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/specia-2013-quest-translation-quality-estimation-framework/)
25. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, et al. 2025a. Fanar: An arabic-centric multimodal generative ai platform. arXiv preprint arXiv:2501.13944. [↗](https://arxiv.org/abs/2501.13944) [our card](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/)
26. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. 2025b. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. [↗](https://arxiv.org/abs/2503.19786)
27. Qwen Team et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2(3). [↗](https://arxiv.org/abs/2407.10671)
28. Unicode Consortium. 2025. Unicode Standard Annex #24: Unicode Script Property. [https://www.unicode.org/reports/tr24/tr24-19.html](https://www.unicode.org/reports/tr24/tr24-19.html). Version 19, published 2025-07-31. [↗](https://www.unicode.org/reports/tr24/tr24-19.html)