---
title: "ARB: A Comprehensive Arabic Multimodal — Ghaboura 2026"
description: "Benchmarks that check how multimodal models reason step by step, and not only whether the final answer is right, exist almost entirely in English."
authors: ["Sara Ghaboura", "Shubham Patle", "Ketan More", "Wafa Hamad Mohamed Alghallabi", "Omkar Thawakar", "Jorma Laaksonen", "Hisham Cholakkal", "Salman Khan", "Rao Anwer"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.723/"
doi: "10.63317/2gyndds6s2us"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ghaboura-2026-arb-comprehensive-arabic-multimodal-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ghaboura-2026-arb-comprehensive-arabic-multimodal-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/) / ARB: A Comprehensive Arabic Multimodal Reasoning Benchmark

# ARB: A Comprehensive Arabic Multimodal Reasoning Benchmark

**Authors:** [Sara Ghaboura](https://md.research.elamur.ai/free/en/authors/sara-ghaboura/), Shubham Patle, Ketan More, Wafa Hamad Mohamed Alghallabi, Omkar Thawakar, Jorma Laaksonen, [Hisham Cholakkal](https://md.research.elamur.ai/free/en/authors/hisham-cholakkal/), [Salman Khan](https://md.research.elamur.ai/free/en/authors/salman-khan/), [Rao Anwer](https://md.research.elamur.ai/free/en/authors/rao-muhammad-anwer/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.723/) · [PDF](https://aclanthology.org/2026.lrec-1.723.pdf) · [DOI 10.63317/2gyndds6s2us](https://doi.org/10.63317/2gyndds6s2us) · [OpenAlex](https://openalex.org/W4416453791)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 140884 papers whose bibliographies we read

## Summary
Benchmarks that check how multimodal models reason step by step, and not only whether the final answer is right, exist almost entirely in English. Arabic has been left out, even though it has a large speaker base and a distinctive culture. The authors build ARB, which they describe as the first Arabic benchmark of stepwise reasoning over both text and images. It holds 2,219 samples across 11 domains, including OCR, charts, maths, culture, medicine, history and remote sensing. The samples come with more than 8K reasoning steps, each tied to an explicit action.

The pipeline works in three stages:
- GPT-4o, prompted in Arabic, drafts the reasoning chains.
- The prompts are steered by four curriculum tracks.
- Native-speaker annotators check and fix the chains in a human-in-the-loop loop.

Evaluation combines similarity metrics, an Arabic LLM-as-Judge rubric and inter-annotator agreement analysis. Fifteen open and closed LMMs were tested, among them the GPT-4o family, Gemini, AIN, Gemma 3, InternVL3 and Qwen. All of them still show weak coherence, faithfulness and cultural grounding. Reasoning that looks fluent often turns out to be logically wrong. The resource matters as a diagnostic tool for transparent reasoning in an underrepresented language.

## Key points
- ARB has 2,219 multimodal samples, 11 domains and over 40 subfields (the taxonomy figure lists 49), with 8K+ human-curated steps and actions.
- Five native Arabic speakers reviewed and validated the reasoning chains.
- A 50-sample pilot, judged by annotators and checked with LaBSE, compared GPT-4o and GPT-4o-mini under Arabic and English prompts. GPT-4o with Arabic prompts won.
- The four curricula are General, Scientific/Medical, Computational and Descriptive/Inferential. Curriculum-guided prompting improved reasoning accuracy by 12.8% over unguided prompting in human evaluation.
- The judge rubric is adapted from LlamaV-o1 and scores each criterion from 1 to 10. Its criteria are:
  - informativeness
  - token repetition
  - hallucination
  - redundancy
  - semantic coverage
  - reasoning alignment
  - commonsense
  - missing steps

## Interesting ideas
- Each reasoning step is stored together with the concrete action it performs, so a chain can be audited one move at a time instead of as free prose. — For Mova (answer): School-test and word-problem solvers could output (step, action) pairs, such as "extract quantity" or "compare holdings", which makes every inference in an answer inspectable.
- Prompts for generating explanations are routed through a small set of reasoning "curricula" chosen per domain, which measurably beat a single generic prompt. — For Mova (learn): LLM teacher tiers could receive curriculum-specific instructions (computational vs. descriptive vs. commonsense) when producing silver explanations, raising the quality of teacher output.
- The judge rubric breaks a chain into separate failure types, such as hallucination, redundancy, missing step and semantic coverage, rather than giving one overall score. — For Mova (learn): Evaluation of explanation traces could report these error classes separately, showing whether a regression comes from invented facts or from skipped inferences.
- Generation, judging and evaluation all stay in the target language to avoid cross-lingual bias in the judge. — For Mova (back): When testing text → graph → text or translation through the meta-world, quality could be judged in the output language itself instead of after back-translation into English.
- A cheap pilot run with human ratings plus an embedding check (LaBSE) picks the generator setup before scaling up data creation. — For Mova (learn): Before mass-producing silver data, a small pilot could compare teacher settings, scored by humans and an automatic similarity metric, to pick the best configuration.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · [Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · [Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ghaboura2026arb,
  title = {{ARB: A Comprehensive Arabic Multimodal Reasoning Benchmark}},
  author = {Ghaboura, Sara and Patle, Shubham and More, Ketan and Alghallabi, Wafa Hamad Mohamed and Thawakar, Omkar and Laaksonen, Jorma and Cholakkal, Hisham and Khan, Salman and Anwer, Rao},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2gyndds6s2us},
  url = {https://doi.org/10.63317/2gyndds6s2us}
}
```

- [← previous A Systematic Comparison of Parameter-Efficient Fine-Tuning Techniques for Low-Resource Neural Machine Translation: Evidence from Indigenous Languages of the Americas](https://md.research.elamur.ai/free/en/papers/stackhouse-2026-systematic-comparison-parameter-efficient-fine/)
- [next → Are Small Language Models the Silver Bullet to Low-Resource Languages Machine Translation?](https://md.research.elamur.ai/free/en/papers/song-2026-are-small-language-models-silver/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/)

## References

55 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mohammed Al-Maghrabi Allam-Research. 2025. Allam-thinking: Arabic large language model with enhanced reasoning capabilities. [https://huggingface.co/almaghrabima/ALLaM-Thinking](https://huggingface.co/almaghrabima/ALLaM-Thinking). [↗](https://huggingface.co/almaghrabima/ALLaM-Thinking)
2. Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, and Ajay Divakaran. 2024a. Measuring and improving chain-of-thought reasoning in vision-language models. In NAACL-HLT. [find](https://scholar.google.com/scholar?q=Yangyi+Chen%2C+Karan+Sikka%2C+Michael+Cogswell%2C+Heng+Ji%2C+and+Ajay+Divakaran%2E+2024a%2E+Measuring+and+improving+chain%2Dof%2Dthought+reasoning+in+vision%2Dlanguage+models%2E+In+NAACL%2DHLT%2E)
3. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. 2024b. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271. [↗](https://arxiv.org/abs/2412.05271) [our card](https://md.research.elamur.ai/free/en/papers/chen-2024-expanding-performance-boundaries-open/)
4. Cohere-Labs. 2025. Aya vision 8b: A multilingual vision-language model. [https://huggingface.co/CohereForAI/](https://huggingface.co/CohereForAI/) aya-vision-8b. Accessed: 2025-05-03. [↗](https://huggingface.co/CohereForAI/)
5. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. [↗](https://arxiv.org/abs/2507.06261) [our card](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/)
6. Google DeepMind. 2024. Gemini 2.0 flash thinking: Unlocking transparent [https://deepmind.reasoning](https://deepmind.reasoning) in ai. google/technologies/gemini/ Accessed: 2025-05-flash-thinking/. 03. [↗](https://deepmind.reasoning)
7. Google DeepMind and Google AI. 2025. Gemini 2.5 flash. [find](https://scholar.google.com/scholar?q=Google+DeepMind+and+Google+AI%2E+2025%2E+Gemini+2%2E5+flash%2E)
8. Fanar-Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, et al. 2025. Fanar: An arabic-centric multimodal generative ai platform. arXiv preprint arXiv:2501.13944. [↗](https://arxiv.org/abs/2501.13944) [our card](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/)
9. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020. Language-agnostic bert sentence embedding. arXiv preprint arXiv:2007.01852. [↗](https://arxiv.org/abs/2007.01852) [our card](https://md.research.elamur.ai/free/en/papers/feng-2020-language-agnostic-bert-sentence-embedding/)
10. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. [↗](https://arxiv.org/abs/2501.12948) [our card](https://md.research.elamur.ai/free/en/papers/deepseek-2025-ai-deepseek-r1-incentivizing-reasoning-capability/)
11. Ahmed Heakl, Sara Ghaboura, Omkar Thawkar, Fahad Shahbaz Khan, Hisham Cholakkal, Rao Muhammad Anwer, and Salman Khan. 2025. Ain: The arabic inclusive large multimodal model. arXiv preprint arXiv:2502.00094. [↗](https://arxiv.org/abs/2502.00094) [our card](https://md.research.elamur.ai/free/en/papers/heakl-2025-ain-arabic-inclusive-large-multimodal/)
12. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720. [↗](https://arxiv.org/abs/2412.16720) [our card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/)
13. Klaus Krippendorff. 2018. Content analysis: An introduction to its methodology. Sage publications. [find](https://scholar.google.com/scholar?q=Klaus+Krippendorff%2E+2018%2E+Content+analysis%3A+An+introduction+to+its+methodology%2E+Sage+publications%2E)
14. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81. [find](https://scholar.google.com/scholar?q=Chin%2DYew+Lin%2E+2004%2E+Rouge%3A+A+package+for+automatic+evaluation+of+summaries%2E+In+Text+summarization+branches+out%2C+pages+74%E2%80%9381%2E) [our card](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/)
15. Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. 2025a. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.06520. [↗](https://arxiv.org/abs/2503.06520)
16. Yuqi Liu, Tianyuan Qu, Zhisheng Zhong, Bohao Peng, Shu Liu, Bei Yu, and Jiaya Jia. 2025b. Visionreasoner: Unified visual perception and reasoning via reinforcement learning. arXiv preprint arXiv:2505.12081. [↗](https://arxiv.org/abs/2505.12081)
17. Meta-AI. 2025. Llama-4-scout-17b-16e-instruct. [https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct). Accessed: 2025-05-03. [↗](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct)
18. Ahmad Mustapha, Hadi Al-Khansa, Hadi Al-Mubasher, Aya Mourad, Ranam Hamoud, Hasan El-Husseini, Marwah Al-Sakkaf, and Mariette Awad. 2024. Arastem: A native arabic multiple choice question benchmark for evaluating llms knowledge in stem subjects. arXiv preprint arXiv:2501.00559. [↗](https://arxiv.org/abs/2501.00559) [our card](https://md.research.elamur.ai/free/en/papers/mustapha-2024-arastem-native-arabic-multiple-choice/)
19. OpenAI. 2024a. Gpt-4o mini: Advancing cost-efficient intelligence. Accessed: 2025-05-03. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024a%2E+Gpt%2D4o+mini%3A+Advancing+cost%2Defficient+intelligence%2E+Accessed%3A+2025%2D05%2D03%2E)
20. OpenAI. 2024b. Gpt-4o system card. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024b%2E+Gpt%2D4o+system+card%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)
21. OpenAI. 2025a. GPT-5 model re-[https://openai.com/index/lease](https://openai.com/index/lease). introducing-gpt-5/. Accessed: 2025-10-24. [↗](https://openai.com/index/lease)
22. OpenAI. 2025b. Introducing gpt-4.1 in the api. Accessed: 2025-05-03. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025b%2E+Introducing+gpt%2D4%2E1+in+the+api%2E+Accessed%3A+2025%2D05%2D03%2E)
23. OpenAI. 2025c. Openai o3 and o4-mini system card. Accessed: 2025-05-03. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025c%2E+Openai+o3+and+o4%2Dmini+system+card%2E+Accessed%3A+2025%2D05%2D03%2E)
24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. [↗](https://doi.org/10.3115/1073083.1073135) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
25. Pinterest. 2025. Pinterest platform. [https://www.pinterest.com/](https://www.pinterest.com/). [↗](https://www.pinterest.com/)
26. Qwen-Team. 2025. Qwen2.5-vl. Accessed: 2025-05-03. [find](https://scholar.google.com/scholar?q=Qwen%2DTeam%2E+2025%2E+Qwen2%2E5%2Dvl%2E+Accessed%3A+2025%2D05%2D03%2E)
27. Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. 2024. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. Advances in Neural Information Processing Systems, 37:8612–8642. [find](https://scholar.google.com/scholar?q=Hao+Shao%2C+Shengju+Qian%2C+Han+Xiao%2C+Guanglu+Song%2C+Zhuofan+Zong%2C+Letian+Wang%2C+Yu+Liu%2C+and+Hongsheng+Li%2E+2024%2E+Visual+cot%3A+Advancing+multi%2Dmodal+language+models+with+a+comprehensive+dataset+and+benchmark+)
28. Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. Synthetic prompting: generating chain-of-thought demonstrations for large language models. In Proceedings of the 40th International Conference on Machine Learning, pages 30706– 30775. [find](https://scholar.google.com/scholar?q=Zhihong+Shao%2C+Yeyun+Gong%2C+Yelong+Shen%2C+Minlie+Huang%2C+Nan+Duan%2C+and+Weizhu+Chen%2E+2023%2E+Synthetic+prompting%3A+generating+chain%2Dof%2Dthought+demonstrations+for+large+language+models%2E+In+Proceedings+of+the+4)
29. Qwen Team. 2025. Qwen3 technical report. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2025%2E+Qwen3+technical+report%2E)
30. Team-Gemma, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. [↗](https://arxiv.org/abs/2503.19786)
31. Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. 2025. Llamav-o1: Rethinking step-by-step visual reasoning in llms. arXiv preprint arXiv:2501.06186. [↗](https://arxiv.org/abs/2501.06186)
32. Emily Vaillancourt and Christopher Thompson. 2024. Instruction tuning on large language models to improve reasoning performance. Authorea Preprints. [find](https://scholar.google.com/scholar?q=Emily+Vaillancourt+and+Christopher+Thompson%2E+2024%2E+Instruction+tuning+on+large+language+models+to+improve+reasoning+performance%2E+Authorea+Preprints%2E)
33. Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2609–2634. [find](https://scholar.google.com/scholar?q=Lei+Wang%2C+Wanyu+Xu%2C+Yihuai+Lan%2C+Zhiqiang+Hu%2C+Yunshi+Lan%2C+Roy+Ka%2DWei+Lee%2C+and+Ee%2DPeng+Lim%2E+2023%2E+Plan%2Dand%2Dsolve+prompting%3A+Improving+zero%2Dshot+chain%2Dof%2Dthought+reasoning+by+large+language+models%2E+In+Pr)
34. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. [↗](https://arxiv.org/abs/2203.11171) [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)
35. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+Fei+Xia%2C+Ed+Chi%2C+Quoc+V+Le%2C+Denny+Zhou%2C+et+al%2E+2022%2E+Chain%2Dof%2Dthought+prompting+elicits+reasoning+in+large+language+models%2E+Advances+in+neural+i) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
36. Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. 2025. Llava-cot: Let vision language models reason step-by-step. [find](https://scholar.google.com/scholar?q=Guowei+Xu%2C+Peng+Jin%2C+Hao+Li%2C+Yibing+Song%2C+Lichao+Sun%2C+and+Li+Yuan%2E+2025%2E+Llava%2Dcot%3A+Let+vision+language+models+reason+step%2Dby%2Dstep%2E)
37. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+Advances+in+neural+inf)
38. Li Yujian and Liu Bo. 2007. A normalized levenshtein distance metric. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6):1091–1095. [find](https://scholar.google.com/scholar?q=Li+Yujian+and+Liu+Bo%2E+2007%2E+A+normalized+levenshtein+distance+metric%2E+IEEE+Transactions+on+Pattern+Analysis+and+Machine+Intelligence%2C+29%286%29%3A1091%E2%80%931095%2E)
39. Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. 2024. Improve vision language model chain-of-thought reasoning. arXiv preprint arXiv:2410.16198. [↗](https://arxiv.org/abs/2410.16198)
40. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. [↗](https://arxiv.org/abs/1904.09675)
41. Language Resource References Alwajih, Fakhraddin and Nagoudi, El Moatez Billah and Bhatia, Gagan and Mohamed, Abdelrahman and Abdul-Mageed, Muhammad. 2024. Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks. arXiv. [find](https://scholar.google.com/scholar?q=Language+Resource+References+Alwajih%2C+Fakhraddin+and+Nagoudi%2C+El+Moatez+Billah+and+Bhatia%2C+Gagan+and+Mohamed%2C+Abdelrahman+and+Abdul%2DMageed%2C+Muhammad%2E+2024%2E+Peacock%3A+A+Family+of+Arabic+Multimodal+Large)
42. Bisong, Ekaba and Bisong, Ekaba. 2019. Matplotlib and seaborn. Springer. [find](https://scholar.google.com/scholar?q=Bisong%2C+Ekaba+and+Bisong%2C+Ekaba%2E+2019%2E+Matplotlib+and+seaborn%2E+Springer%2E)
43. Das, Rocktim Jyoti and Hristov, Simeon Emilov and Li, Haonan and Dimitrov, Dimitar and Koychev, Ivan and Nakov, Preslav. 2024. EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Das%2C+Rocktim+Jyoti+and+Hristov%2C+Simeon+Emilov+and+Li%2C+Haonan+and+Dimitrov%2C+Dimitar+and+Koychev%2C+Ivan+and+Nakov%2C+Preslav%2E+2024%2E+EXAMS%2DV%3A+A+Multi%2DDiscipline+Multilingual+Multimodal+Exam+Benchmark+for+Ev)
44. Jevgenij Gamper and Nasir Rajpoot. 2021. Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles. arXiv. [find](https://scholar.google.com/scholar?q=Jevgenij+Gamper+and+Nasir+Rajpoot%2E+2021%2E+Multiple+Instance+Captioning%3A+Learning+Representations+from+Histopathology+Textbooks+and+Articles%2E+arXiv%2E)
45. Ghaboura, Sara and Heakl, Ahmed and Thawakar, Omkar and Alharthi, Ali Husain Salem Abdulla and Riahi, Ines and Radman, Abduljalil and Laaksonen, Jorma and Khan, Fahad Shahbaz and Khan, Salman and Anwer, Rao Muhammad. 2025a. CAMEL-Bench: A Comprehensive Arabic LMM Benchmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ghaboura%2C+Sara+and+Heakl%2C+Ahmed+and+Thawakar%2C+Omkar+and+Alharthi%2C+Ali+Husain+Salem+Abdulla+and+Riahi%2C+Ines+and+Radman%2C+Abduljalil+and+Laaksonen%2C+Jorma+and+Khan%2C+Fahad+Shahbaz+and+Khan%2C+Salman+and+Anwe)
46. Ghaboura, Sara and More, Ketan Pravin and Thawkar, Ritesh and Ghallabi, Wafa Al and Thawakar, Omkar and Khan, Fahad Shahbaz and Cholakkal, Hisham and Khan, Salman and Anwer, Rao Muhammad. 2025b. Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifacts. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ghaboura%2C+Sara+and+More%2C+Ketan+Pravin+and+Thawkar%2C+Ritesh+and+Ghallabi%2C+Wafa+Al+and+Thawakar%2C+Omkar+and+Khan%2C+Fahad+Shahbaz+and+Cholakkal%2C+Hisham+and+Khan%2C+Salman+and+Anwer%2C+Rao+Muhammad%2E+2025b%2E+Time+)
47. He, Xuehai and Zhang, Yichen and Mou, Luntian and Xing, Eric and Xie, Pengtao. 2020. Pathvqa: 30000+ questions for medical visual question answering. arXiv. [find](https://scholar.google.com/scholar?q=He%2C+Xuehai+and+Zhang%2C+Yichen+and+Mou%2C+Luntian+and+Xing%2C+Eric+and+Xie%2C+Pengtao%2E+2020%2E+Pathvqa%3A+30000%2B+questions+for+medical+visual+question+answering%2E+arXiv%2E)
48. Kadaoui, Karima and Atwany, Hanin and Al-Ali, Hamdan and Mohamed, Abdelrahman and Mekky, Ali and Tilga, Sergei and Fedorova, Natalia and Artemova, Ekaterina and Aldarmaki, Hanan and Kementchedjhieva, Yova. 2025. JEEM: Vision-Language Understanding in Four Arabic Dialects. arXiv. [find](https://scholar.google.com/scholar?q=Kadaoui%2C+Karima+and+Atwany%2C+Hanin+and+Al%2DAli%2C+Hamdan+and+Mohamed%2C+Abdelrahman+and+Mekky%2C+Ali+and+Tilga%2C+Sergei+and+Fedorova%2C+Natalia+and+Artemova%2C+Ekaterina+and+Aldarmaki%2C+Hanan+and+Kementchedjhieva%2C+)
49. Kembhavi, Aniruddha and Salvato, Mike and Kolve, Eric and Seo, Minjoon and Hajishirzi, Hannaneh and Farhadi, Ali. 2016. A diagram is worth a dozen images. Springer. [find](https://scholar.google.com/scholar?q=Kembhavi%2C+Aniruddha+and+Salvato%2C+Mike+and+Kolve%2C+Eric+and+Seo%2C+Minjoon+and+Hajishirzi%2C+Hannaneh+and+Farhadi%2C+Ali%2E+2016%2E+A+diagram+is+worth+a+dozen+images%2E+Springer%2E) [our card](https://md.research.elamur.ai/free/en/papers/kembhavi-2016-diagram-worth-dozen-images/)
50. Microsoft Corporation. 2024. Microsoft Excel. Microsoft Corporation. Version 16.0. [find](https://scholar.google.com/scholar?q=Microsoft+Corporation%2E+2024%2E+Microsoft+Excel%2E+Microsoft+Corporation%2E+Version+16%2E0%2E)
51. Mousi, Basel and Durrani, Nadir and Ahmad, Fatema and Hasan, Md Arid and Hasanain, Maram and Kabbani, Tameem and Dalvi, Fahim and Chowdhury, Shammur Absar and Alam, Firoj. 2024. Aradice: Benchmarks for dialectal and cultural capabilities in llms. arXiv. [find](https://scholar.google.com/scholar?q=Mousi%2C+Basel+and+Durrani%2C+Nadir+and+Ahmad%2C+Fatema+and+Hasan%2C+Md+Arid+and+Hasanain%2C+Maram+and+Kabbani%2C+Tameem+and+Dalvi%2C+Fahim+and+Chowdhury%2C+Shammur+Absar+and+Alam%2C+Firoj%2E+2024%2E+Aradice%3A+Benchmarks+fo)
52. Nawaz, Umair and Muhammad, Awais and Gani, Hanan and Naseer, Muzammal and Khan, Fahad Shahbaz and Khan, Salman and Anwer, Rao. 2025. AgriCLIP: Adapting CLIP for Agriculture and Livestock via Domain-Specialized Cross-Model Alignment. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nawaz%2C+Umair+and+Muhammad%2C+Awais+and+Gani%2C+Hanan+and+Naseer%2C+Muzammal+and+Khan%2C+Fahad+Shahbaz+and+Khan%2C+Salman+and+Anwer%2C+Rao%2E+2025%2E+AgriCLIP%3A+Adapting+CLIP+for+Agriculture+and+Livestock+via+Domain%2DSp)
53. Noman, Mubashir and Ahsan, Noor and Naseer, Muzammal and Cholakkal, Hisham and Anwer, Rao Muhammad and Khan, Salman H and Khan, Fahad Shahbaz. 2024. CDChat: A Large Multimodal Model for Remote Sensing Change Description. [find](https://scholar.google.com/scholar?q=Noman%2C+Mubashir+and+Ahsan%2C+Noor+and+Naseer%2C+Muzammal+and+Cholakkal%2C+Hisham+and+Anwer%2C+Rao+Muhammad+and+Khan%2C+Salman+H+and+Khan%2C+Fahad+Shahbaz%2E+2024%2E+CDChat%3A+A+Large+Multimodal+Model+for+Remote+Sensing)
54. Python Software Foundation. 2024. Python Language Reference, version 3.12. [find](https://scholar.google.com/scholar?q=Python+Software+Foundation%2E+2024%2E+Python+Language+Reference%2C+version+3%2E12%2E)
55. Sadallah, Abdelrahman and Tonga, Junior Cedric and Almubarak, Khalid and Almheiri, Saeed and Atif, Farah and Qwaider, Chatrine and Kadaoui, Karima and Shatnawi, Sara and Alesh, Yaser and Koto, Fajri. 2025. Commonsense Reasoning in Arab Culture. Association for Computational Linguistics. scottgeng00. 2025. NaBirds. [Accessed 10-10-2025]. [find](https://scholar.google.com/scholar?q=Sadallah%2C+Abdelrahman+and+Tonga%2C+Junior+Cedric+and+Almubarak%2C+Khalid+and+Almheiri%2C+Saeed+and+Atif%2C+Farah+and+Qwaider%2C+Chatrine+and+Kadaoui%2C+Karima+and+Shatnawi%2C+Sara+and+Alesh%2C+Yaser+and+Koto%2C+Fajri%2E+)