---
title: "On LLM Prompting Techniques for Arabic Language Arithmetic"
description: "The authors build three Arabic datasets. 97.65% accuracy on AGSM8K. 23 references. Among them: 0."
authors: ["Reem Alenezi", "Ayed Atallah Salman"]
year: 2026
venue: "The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks"
source: "https://aclanthology.org/2026.osact-1.13/"
doi: "10.63317/52s6fyvjghcs"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/alenezi-2026-llm-prompting-techniques-arabic-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alenezi-2026-llm-prompting-techniques-arabic-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/43/) / On LLM Prompting Techniques for Arabic Language Arithmetic Reasoning

# On LLM Prompting Techniques for Arabic Language Arithmetic Reasoning

**Authors:** Reem Alenezi and Ayed Atallah Salman · **Year:** 2026 · **Venue:** [The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks](https://md.research.elamur.ai/free/en/venues/osact/) ([Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks 2026](https://md.research.elamur.ai/free/en/venues/osact/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.osact-1.13/) · [PDF](https://aclanthology.org/2026.osact-1.13.pdf) · [DOI 10.63317/52s6fyvjghcs](https://doi.org/10.63317/52s6fyvjghcs) · [OpenAlex](https://openalex.org/W7166518814)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks how well large language models solve arithmetic word problems written in Arabic, a language with little benchmark data for this task. The authors build three Arabic datasets. AGSM8K is a translation of GSM8K made with ChatGPT and then checked by hand and by a language reviewer. Qudurat comes from Kuwait University aptitude-test items, with a mathematics specialist writing out missing solutions. ArabicMWPs combines translated items from AddSub and MultiArith. On these datasets they test six LLMs, both closed and open, with three prompting strategies: Manual CoT, Zero-shot CoT and Self-consistency. Scoring has two parts. Accuracy is measured on the final answer, and the generated reasoning is compared with reference solutions using BERTScore. The best setup is GPT-4o with Self-consistency, which reaches 97.65% on AGSM8K. The Arabic-specific ALLaM does much worse on the harder data. Fine-tuning on Arabic maths data clearly helps open-source models. Open models also show a large gap between English and Arabic. The work supplies benchmarks and baselines for Arabic mathematical reasoning.

## Key points
- Three new datasets:
  - AGSM8K: 7,473 training and 1,319 test problems needing 2–8 steps.
  - Qudurat: 140 Kuwait University aptitude problems needing 1–8 steps.
  - ArabicMWPs: 725 problems, 265 from AddSub and 460 from MultiArith.
- GPT-4o with Self-consistency scores 97.65% accuracy on AGSM8K. Its BERTScore on the reasoning text is precision 71.94%, recall 71.31% and F1 71.50%.
- ALLaM reaches 84.41% on ArabicMWPs but only 43.97% on AGSM8K.
- Self-consistency was not run for ALLaM because sampling many reasoning paths was too expensive and too slow.
- Fine-tuning open-source models on Arabic mathematical data improved their results substantially.

## Interesting ideas
- The evaluation scores both the final number and the reasoning text, using BERTScore against worked reference solutions, so a right answer reached by a wrong route can still be detected.
- Self-consistency samples several independent reasoning paths and keeps the answer that most of them agree on, which reduces one-off logical slips.
- A benchmark is built by machine-translating an existing one into a JSONL format, followed by manual checks of mathematical consistency and then a separate language reviewer.
- The three datasets are graded by difficulty, from addition-and-subtraction-only items up to problems needing 8 steps, which shows where a model's ability breaks down; ALLaM's sharp drop between datasets is an example.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): Mova's maths word-problem solver could be scored on its explanation trace against gold step lists as well as on the answer, which would separate lucky guesses from sound derivations.
- **I2** (learn): the same majority idea fits silver-data agreement, where several LLM teacher runs or witness parsers vote and only answers with strong agreement become training data.
- **I3** (learn): this layered pipeline (machine draft, logic check, fluency check) could be reused to create or port evaluation sets cheaply while keeping errors under control.
- **I4** (answer): splitting word-problem tests by step count and operation type would show at which reasoning depth Mova's world model stops tracking quantities.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Kuwait University](https://md.research.elamur.ai/free/en/universities/kuwait-university/) — University, Kuwait City, Kuwait · 11 works · 147 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 321 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works

## Cite

Cite as: Reem Alenezi and Ayed Atallah Salman. 2026. On LLM Prompting Techniques for Arabic Language Arithmetic Reasoning. The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{alenezi2026llm,
  title = {{On LLM Prompting Techniques for Arabic Language Arithmetic Reasoning}},
  author = {Reem Alenezi and Ayed Atallah Salman},
  booktitle = {The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks},
  year = {2026},
  doi = {10.63317/52s6fyvjghcs},
  url = {https://doi.org/10.63317/52s6fyvjghcs}
}
```

- [← previous On extralinguistic motivation behind food-specific metaphorical transfers in the history of English](https://md.research.elamur.ai/free/en/papers/kowalczyk-2026-extralinguistic-motivation-behind-food-specific/)
- [next → On Natural Language Stringsets, Senses, and Generative Capacity](https://md.research.elamur.ai/free/en/papers/radzinski-2026-natural-language-stringsets-senses-generative/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/43/)

## References

23 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahn, J., Verma, R., Lou, R., Liu, D., Zhang, R., and Yin, W. (2024). [Large language models for mathematical reasoning: Progresses and challenges](https://md.research.elamur.ai/free/en/papers/ahn-2024-large-language-models-mathematical-reasoning/). arXiv:2402.00157. [↗](https://arxiv.org/abs/2402.00157)
2. Albahli, S. (2025). An advanced natural language processing framework for Arabic named entity recognition: A novel approach to handling morphological richness and nested entities. Applied Sciences, 15(6), 3073. [↗](https://doi.org/10.3390/app15063073)
3. Alghamdi, R., Liang, Z., and Zhang, X. (2022). [ArMath: A dataset for solving Arabic math word problems](https://md.research.elamur.ai/free/en/papers/alghamdi-2022-armath-dataset-solving-arabic-math/). In Proceedings of the Thirteenth Language Resources and Evaluation Conference (LREC 2022), pages 351–362.
4. Al-Khalifa, S., and Al-Khalifa, H. (2024). [The Qiyas benchmark: Measuring ChatGPT mathematical and language understanding in Arabic](https://md.research.elamur.ai/free/en/papers/khalifa-2024-qiyas-benchmark-measuring-chatgpt-mathematical/). arXiv:2407.00146. [↗](https://arxiv.org/abs/2407.00146)
5. Allam, A., Ahmed, S., Hamdi, A., and Mohammed, A. (2025). [Arabic large language models for medical text generation](https://md.research.elamur.ai/free/en/papers/allam-2025-arabic-large-language-models-medical/). In Proceedings of the 4th International Conference on Computer Technologies (ICCTech) (pp. 1–6).
6. Amatriain, X. (2024). Prompt design and engineering: Introduction and advanced methods. arXiv:2401.14423. [↗](https://arxiv.org/abs/2401.14423)
7. Bari, M. S., Alnumay, Y., Alzahrani, N. A., Alotaibi, N. M., Alyahya, H. A., AlRashed, S. et al. (2024). [Allam: Large language models for Arabic and English](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/). arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390)
8. Basem, M., Oshallah, I., Hamdi, A., and Mohamed, A. (2025). [Few-shot prompting for extractive Quranic QA with instruction-tuned LLMs](https://md.research.elamur.ai/free/en/papers/basem-2025-few-shot-prompting-extractive-quranic/). In Proceedings of the Intelligent Methods, Systems, and Applications (IMSA) (pp. 24–29).
9. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P. et al. (2020). [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in Neural Information Processing Systems, 33, 1877–1901. [↗](https://doi.org/10.48550/arxiv.2005.14165)
10. Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L. et al. (2021). [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv:2110.14168. [↗](https://arxiv.org/abs/2110.14168)
11. Diao, S., Wang, P., Lin, Y., Pan, R., Liu, X., and Zhang, T. (2023). Active prompting with chain-of-thought for large arXiv:2302.12246. language models. [↗](https://arxiv.org/abs/2302.12246)
12. Hosseini, M. J., Hajishirzi, H., Etzioni, O., and Kushman, N. (2014). [Learning to solve arithmetic word problems with verb categorization](https://md.research.elamur.ai/free/en/papers/hosseini-2014-learning-solve-arithmetic-word-problems/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 523–533).
13. Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. (2022). [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). arXiv:2205.11916. Kuwait University. Math Test Samples. Available at: https://cem.ku.edu.kw/mathsample [↗](https://arxiv.org/abs/2205.11916)
14. Kwon, S. Y., Bhatia, G., Nagoud, E. M. B., and Abdul-Mageed, M. (2023). ChatGPT for Arabic grammatical error correction. arXiv:2308.04492. [↗](https://arxiv.org/abs/2308.04492)
15. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. (2023). [Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/). ACM Computing Surveys, 55(9), 1–35. [↗](https://doi.org/10.1145/3560815)
16. Lu, P., Qiu, L., Yu, W., Welleck, S., and Chang, K.-W. (2022). A survey of deep learning for mathematical reasoning. arXiv:2212.10535. [↗](https://arxiv.org/abs/2212.10535)
17. Roy, S., and Roth, D. (2016). [Solving general arithmetic word problems](https://md.research.elamur.ai/free/en/papers/roy-2015-solving-general-arithmetic-word-problems/). arXiv:1608.01413. Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. [↗](https://arxiv.org/abs/1608.01413)
18. K. W., and Lim, E. P. (2023). Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv:2305.04091. [↗](https://arxiv.org/abs/2305.04091)
19. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S. et al. (2022). [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv:2203.11171. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F. et al. Chain-of-thought prompting elicits reasoning in large language models. arXiv:2201.11903. [↗](https://arxiv.org/abs/2203.11171)
20. World Population Review (2024). Arabic speaking countries. Available at: [https://worldpopulationreview.com/country-rankings/arabic-speaking-countries](https://worldpopulationreview.com/country-rankings/arabic-speaking-countries) [↗](https://worldpopulationreview.com/country-rankings/arabic-speaking-countries)
21. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. (2023). Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 11809–11822. Zhang, Q., Chen, M., Bukharin, A., Karampatziakis, N., He, P., Cheng, Y. et al. [find](https://scholar.google.com/scholar?q=Yao%2C+S%2E%2C+Yu%2C+D%2E%2C+Zhao%2C+J%2E%2C+Shafran%2C+I%2E%2C+Griffiths%2C+T%2E%2C+Cao%2C+Y%2E%2C+and+Narasimhan%2C+K%2E+%282023%29%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+Advances+in+Neural+Information+Proce)
22. (2023). [AdaLoRA: Adaptive budget allocation for parameter-efficient fine-tuning](https://md.research.elamur.ai/free/en/papers/zhang-2023-adalora-adaptive-budget-allocation-parameter/). arXiv:2303.10512. [↗](https://arxiv.org/abs/2303.10512)
23. Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. (2019). BERTScore: Evaluating text generation with BERT. arXiv:1904.09675. [↗](https://arxiv.org/abs/1904.09675)

*Updated 1 November 2026*