---
title: "Tool Calling for Arabic LLMs — paper by Ersoy et al. (2025)"
description: "Most tool-calling resources and research target English, so it is unclear how to give an Arabic LLM… 14 references, cited by 1 paper here."
authors: ["Asım Ersoy", "Enes Altinisik", "Kareem Mohamed Darwish", "Husrev Taha Sencar"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2025.arabicnlp-main.28/"
doi: "10.18653/v1/2025.arabicnlp-main.28"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ersoy-2025-tool-calling-arabic-llms-data/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ersoy-2025-tool-calling-arabic-llms-data/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/13/) / Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning

# Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning

**Authors:** Asım Ersoy, [Enes Altinisik](https://md.research.elamur.ai/free/en/authors/enes-altinisik/), [Kareem Mohamed Darwish](https://md.research.elamur.ai/free/en/authors/kareem-darwish/) and [Husrev Taha Sencar](https://md.research.elamur.ai/free/en/authors/husrev-taha-sencar/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-main.28/) · [PDF](https://aclanthology.org/2025.arabicnlp-main.28.pdf) · [DOI 10.18653/v1/2025.arabicnlp-main.28](https://doi.org/10.18653/v1/2025.arabicnlp-main.28) · [OpenAlex](https://openalex.org/W4416037647)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/ersoy-2025-tool-calling-arabic-llms-data/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most tool-calling resources and research target English, so it is unclear how to give an Arabic LLM reliable function-calling skills. The authors ask three questions: whether Arabic tool-calling training data is needed or cross-lingual transfer is enough, whether general instruction tuning helps tool calling, and whether fine-tuning on a few priority tools is worth the effort. They machine-translated Glaive and xLAM into Arabic with Gemini-2.5-Flash, producing parallel English/Arabic splits. They also built two new sets: CustomTools, a synthetic set of 8 tools with positive and negative cases, and IslamicRAGTool, derived from real logs of the Fanar Islamic QA service. Using LLaMA-Factory, they ran five SFT experiments on Fanar-1-9B and Fanar-1-9B-Instruct. Each model was scored on tool-name precision and recall and on a strict argument-correctness metric (ArgA). Three findings stand out: bilingual data teaches the model to write arguments in the right language, starting from the instruction-tuned model helps, and adding examples for the target tools pushes argument accuracy close to perfect. These results give practical guidance for building agents in languages other than English.

## Key points
- **Released resources:**
  - Arabic versions of Glaive (972 unique tools) and xLAM (3,179 tools), plus CustomTools and IslamicRAGTool.
  - Training sets reach tens of thousands of examples, e.g. 58,999 xLAM positive samples per language.
  - A tool-calling fine-tuned Fanar model.
- **Training setup:**
  - Cosine learning-rate schedule with a peak of 5.0 × 10⁻⁷.
  - Batch size of 640.
- **Metrics:**
  - Per-tool precision and recall, combined as an average weighted by each tool's test support.
  - ArgA, which counts a call as correct only if the function name and every argument value are right.
- **Error analysis** (249 cases sampled from 7,211 argument errors):
  - Paraphrased arguments: 50.2%.
  - Arguments in the wrong language: 38.2%, and the leading cause on Arabic tests at 53.1%.
  - Plainly wrong values: 6.8%.
  - Missing multi-turn context: 4.8%.
- **Gains from data choices:**
  - Bilingual data raised ArgA on CustomTools AR from 0.58 to 0.80.
  - Tool-specific fine-tuning lifted ArgA to about 0.91–0.98 on CustomTools and IslamicRAGTool, from roughly 0.45–0.74 without it.

## Interesting ideas
- Argument failures are split into wrong value, wrong language, paraphrase, and missing context, which separates real misunderstanding from mismatches in surface form.
- The training and test data deliberately mix in negative cases where no call is needed or the needed tool is absent, so the model learns when to abstain.
- Scoring is two-tier: a lenient score weighted by each tool's support for choosing the right tool, plus a strict all-slots-correct score that exposes slot filling as the real bottleneck.
- English and Arabic train/test splits are direct translations of each other, so differences in results reflect language alone and not content.
- The authors propose that very explicit argument descriptions, such as stating the required language or date format, could reduce ambiguity without extra training data.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Mova's evaluation could tag each failure as "meaning wrong" versus "surface form differs from gold", so repair rules target real errors and not formatting noise.
- **I2** (answer): Mova's readers could get test items whose correct output is "not answerable from this text", acting as negative controls against over-eager answering.
- **I3** (frames): frame evaluation could report frame-type precision and recall weighted by frequency alongside an exact-match rate over all semantic roles, showing whether errors come from frame choice or role filling.
- **I4** (back): a parallel test set would let Mova's translation through the meta-world be scored on identical meaning graphs in both languages, isolating losses in generation.
- **I5** (answer): math word-problem solvers could carry explicit answer-format specs (units, rounding, number form), so that correct answers are not marked wrong for surface variation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hamad bin Khalifa University](https://md.research.elamur.ai/free/en/universities/hamad-bin-khalifa-university/) — University, Doha, Qatar · 141 works · 14,175 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Asım Ersoy, Enes Altinisik, Kareem Mohamed Darwish and Husrev Taha Sencar. 2025. Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning. Proceedings of The Third Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ersoy2025tool,
  title = {{Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning}},
  author = {Asım Ersoy, Enes Altinisik, Kareem Mohamed Darwish and Husrev Taha Sencar},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-main.28},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-main.28}
}
```

- [← previous Thinking Nodes at MAHED: A Comparative Study of Multimodal Architectures for Arabic Hateful Meme Detection](https://md.research.elamur.ai/free/en/papers/safwan-2025-thinking-nodes-mahed-comparative-study/)
- [next → Transforming Causal LLM into MLM Encoder for Detecting Social Media Manipulation in Telegram](https://md.research.elamur.ai/free/en/papers/bazdyrev-2025-transforming-causal-llm-into-mlm/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/13/)

## References

14 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ibrahim Abdelaziz, Kinjal Basu, Mayank Agarwal, Sadhana Kumaravel, Matthew Stallone, Rameswar Panda, Yara Rizk, G. P. Shrivatsa Bhargav, Maxwell Crouse, Chulaka Gunasekara, Shajith Ikbal, Sachindra Joshi, Hima Karanam, Vineet Kumar, Asim Munawar, Sumit Neelam, Dinesh Raghu, Udit Sharma, Adriana Meza Soria, and 2 others. 2024. Granite-function calling model: Introducing function calling abilities via multi-task learning of granular tasks. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), Industry Track. Also available at arXiv:2407.00121. [↗](https://arxiv.org/abs/2407.00121)
2. Zhiyuan Chen, Shiqi Shen, Guangyao Shen, Gong Zhi, Xu Chen, and Yankai Lin. 2024. Towards tool use alignment of large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1382–1400. Code and data: [https://github.com/zhiyuanc2001/ToolAlign](https://github.com/zhiyuanc2001/ToolAlign). [↗](https://github.com/zhiyuanc2001/ToolAlign)
3. Shijue Huang, Wanjun Zhong, Jianqiao Lu, Qi Zhu, Jiahui Gao, Weiwen Liu, Yutai Hou, Xingshan Zeng, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruifeng Xu, and Qun Liu. 2024. Planning, creation, usage: Benchmarking llms for comprehensive tool utilization in real-world complex scenarios. In Findings of the Association for Computational Linguistics: ACL 2024, pages 4200–4216. ArXiv:2401.17167. [↗](https://arxiv.org/abs/2401.17167)
4. Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. [Api-bank: A comprehensive benchmark for tool-augmented llms](https://md.research.elamur.ai/free/en/papers/li-2023-api-bank-comprehensive-benchmark-tool/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). ArXiv preprint arXiv:2304.08244. [↗](https://arxiv.org/abs/2304.08244)
5. Qwen Team. 2024. Qwen2.5: A party of foundation models. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024%2E+Qwen2%2E5%3A+A+party+of+foundation+models%2E)
6. Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, Zezhong Wang, Yuxian Wang, Wu Ning, Yutai Hou, Bin Wang, Chuhan Wu, Xinzhi Wang, Yong Liu, Yasheng Wang, and 8 others. 2024a. Toolace: Winning the points of llm function calling. arXiv preprint arXiv:2409.00920. [↗](https://arxiv.org/abs/2409.00920)
7. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. [Llamafactory: Unified efficient fine-tuning of 100+ language models](https://md.research.elamur.ai/free/en/papers/zheng-2024-llamafactory-unified-efficient-fine-tuning/). In ACL. Association for Computational Linguistics.
8. A Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh Murthy, Liangwei Yang, Silvio Savarese, Juan Carlos Niebles, Huan Wang, Shelby Heinecke, and Caiming Xiong. 2024b. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Datasets and Benchmarks Track. ArXiv:2406.18518. [↗](https://arxiv.org/abs/2406.18518)
9. Tula Masterman, Sandi Besen, Mason Sawtell, and Alex Chao. 2024. The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey. arXiv preprint arXiv:2404.11584. Prompts System Prompt for Translating Datasets (ex. Glaive) You are a helpful assistant tasked with translating user queries and function argument values into Arabic, using the descriptions of the tools and their arguments as guidance. Translation Guidelines • Do not translate function names or argument keys — only translate the values inside the arguments. • Do not modify any values that are clearly identifiers. [↗](https://arxiv.org/abs/2404.11584)
10. Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2024. [Gorilla: Large language model connected with massive apis](https://md.research.elamur.ai/free/en/papers/patil-2023-gorilla-large-language-model-connected/). In Advances in Neural Information Processing Systems 37 (NeurIPS 2024). ArXiv:2305.15334. • Preserve the original JSON structure exactly as it is. [↗](https://arxiv.org/abs/2305.15334)
11. Baolin Qin, Yuxuan Wang, Yifan Xu, Yuxiang Meng, Yifan Wang, Zhiyang Teng, Jun Yan, Zhiqiang Wei, Yansong Feng, Zheng Wang, and Dongyan Zhao. 2024. [Toolllm: Facilitating large language models to master 16000+ real-world apis](https://md.research.elamur.ai/free/en/papers/qin-2023-toolllm-facilitating-large-language-models/). In Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS 2024). OpenReview: dHng2O0Jjr.
12. Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, Boxi Cao, and Le Sun. 2023. [Toolalpaca: Generalized tool learning for language models with 3000 simulated cases](https://md.research.elamur.ai/free/en/papers/tang-2023-toolalpaca-generalized-tool-learning-language/). arXiv preprint arXiv:2306.05301. ArXiv:2306.05301. [↗](https://arxiv.org/abs/2306.05301)
13. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, and 23 others. 2025. [Fanar: An arabic-centric multimodal generative ai platform](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/).
14. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805. Expected Output Format Always return a JSON object with the following two keys: • "question": The user translated into Arabic. query, • "function_calls": A list of function call objects, where only the argument values are translated into Arabic. System Prompts for Synthetic Data Generation We do this process in a couple of steps to have the LLM focusing on a few tasks at a time, which yielded better data in our case: \# Step 1: Generating questions that can be answered by the given tools. \# Step 3: Generate an answer and a paraphrase for each answer for every query based on the given function schema Generate N unique English example questions that can be answered using the tools listed below. Ensure the following conditions are met: • Each question should distinct—no repetitions near-duplicates. be or Generate a sample answer and a paraphrase of that answer based on the provided question and the corresponding function. • Distribute the questions equally across all tools. • Separate the answer and the paraphrase using a double newline character. • Format the output as a list of tuples, where each tuple contains: • Both the answer and the paraphrase must be written in English. – The question as a string. – The corresponding tool/function name as a string. • Do not mention or reference the tool or function used in either the answer or the paraphrase. The output should be a tuple containing the question and the related tool: (query, function-name). • Assume that all tools operate correctly and return accurate, expected results without any errors or failures. \# Step 2: Populate the related arguments of the function given query/function-name pairs. • Do not generate or execute any code; your output should only be natural language text. You are given a list of tuples, where each tuple contains: • The paraphrase must include the answer itself, but rephrased with different wording and sentence structure. • A natural language query (i.e., a question or instruction), and • The corresponding function name (i.e., the tool that should be invoked to fulfill the query). Each function has a predefined schema specifying its required arguments. Your task is to extract and populate all required argument values for that function based on the information provided in the query. For each input tuple, return a new tuple in the following format: (query, function-name, arguments-dict) [↗](https://arxiv.org/abs/2312.11805)

*Updated 31 October 2026*