---
title: "HIAST at QIAS 2025 — paper by Hamed et al. (2025)"
description: "The paper describes a team's entry to QIAS 2025, an Arabic multiple-choice benchmark on Islamic… 21 references, cited by 4 papers here."
authors: ["Mohamed Motasim Hamed", "Nada Ghneim", "Riad Sonbol"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks"
source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.122/"
doi: "10.18653/v1/2025.arabicnlp-sharedtasks.122"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hamed-2025-hiast-qias-2025-retrieval-augmented/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hamed-2025-hiast-qias-2025-retrieval-augmented/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/) / HIAST at QIAS 2025: Retrieval-Augmented LLMs with Top-Hit Web Evidence for Arabic Islamic Reasoning QA

# HIAST at QIAS 2025: Retrieval-Augmented LLMs with Top-Hit Web Evidence for Arabic Islamic Reasoning QA

**Authors:** [Mohamed Motasim Hamed](https://md.research.elamur.ai/free/en/authors/mohamed-motasim-hamed/), [Nada Ghneim](https://md.research.elamur.ai/free/en/authors/nada-ghneim/) and Riad Sonbol · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.122/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.122.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.122](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.122) · [OpenAlex](https://openalex.org/W4416037634)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/hamed-2025-hiast-qias-2025-retrieval-augmented/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes a team's entry to QIAS 2025, an Arabic multiple-choice benchmark on Islamic studies. It has two subtasks: inheritance-share reasoning (farāʾiḍ), with items generated from IslamWeb fatwas, and general religious knowledge drawn from classical and modern books. The authors neither fine-tune nor build a vector index. Both systems call Claude Opus 4 through its API.

For inheritance questions, the system sends an Arabic query to the Google Custom Search API and keeps only the first result. It strips HTML and boilerplate, cuts the text to 2000 characters and inserts it into a few-shot Arabic prompt. For knowledge questions, whose sources are mostly not indexed online, retrieval is skipped. Instead, three validation items chosen for topical and difficulty spread serve as exemplars.

The entries placed 4th of 15 (0.895) and 3rd of 10 (0.9259). The broader lesson is that retrieval should follow where the knowledge actually lives. One well-ranked page helps a lot when the source is on the web, but the same lookup hurts when it is not.

## Key points
- On Subtask 1 validation, top-hit retrieval raised Claude Opus 4 from 0.785 (no retrieval) and 0.812 (built-in web search) to 0.924. Gains also appeared for Gemini 2.5 Flash (0.700→0.871), GPT 4.1 Mini (0.580→0.822) and Fanar (0.574→0.645).
- On Subtask 2 validation with Gemini 2.5 Flash, adding web retrieval to few-shot prompting lowered accuracy from 0.875 to 0.805, a drop of about 7%.
- Subtask 1 data: about 20,000 train, 1,000 validation and 1,000 test items, each with six options. The authors ignored the supplied corpus of 3,165 fatwas and searched live instead. Subtask 2 data: 1,400 four-option items built from 25 reference texts.
- Test error was 7.37% when web context was found (923 questions) and 9.09% when it was not (77 questions). Similarity between the question and the retrieved text, measured with Muffakir Embedding, barely tracked correctness (correlation 0.027).
- Settings: temperature 0.0, max_tokens 1000, k = 3 exemplars. Diacritics were kept in search queries, and outputs were normalized to option letters.

## Interesting ideas
- Instead of multi-document retrieval, take only the search engine's first hit and trust its ranking; the near-zero similarity–correctness correlation suggested extra hits would add little.
- Treat retrieval as something that can harm, and decide per task based on where the source material lives, backed by an ablation that showed a 7% drop.
- Split the error rate by whether supporting context was available, which separates knowledge gaps from reasoning failures and estimates the gain possible from better coverage.
- Check whether a retrieval similarity score predicts correctness before using it as a confidence signal; here it did not.
- Pick few-shot exemplars from the validation split for topic and difficulty diversity, which avoids test leakage and reduces bias toward frequent topics.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (world): when grounding an entity against Wikidata or WordNet, try a single best-ranked candidate first and measure whether adding more candidates actually changes answers.
- **I2** (learn): every compiled knowledge source plugged into a reader should face a with/without ablation, so that harmful sources are caught rather than assumed helpful.
- **I3** (answer): bucket wrong answers on school tests and fable QA by whether the world of the text held the needed fact, to see whether fixes belong in knowledge or in inference.
- **I4** (learn): before using embedding or agreement scores to filter silver data, verify that they actually correlate with correctness on a gold sample.
- **I5** (learn): LLM teacher prompts could draw exemplars spread across construction types and difficulty, taken from held-out dev data only.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Arab International University](https://md.research.elamur.ai/free/en/universities/arab-international-university/) — University, Ghabāghib, Syria · 13 works · 46 citations
- [Higher Institute for Applied Sciences and Technology](https://md.research.elamur.ai/free/en/universities/higher-institute-for-applied-sciences-and-technology/) — University, Damascus, Syria · 3 works · 4 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (5): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Mohamed Motasim Hamed, Nada Ghneim and Riad Sonbol. 2025. HIAST at QIAS 2025: Retrieval-Augmented LLMs with Top-Hit Web Evidence for Arabic Islamic Reasoning QA. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hamed2025hiast,
  title = {{HIAST at QIAS 2025: Retrieval-Augmented LLMs with Top-Hit Web Evidence for Arabic Islamic Reasoning QA}},
  author = {Mohamed Motasim Hamed, Nada Ghneim and Riad Sonbol},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.122},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.122}
}
```

- [← previous Generative AI and linguistic diversity in academic writing and publishing: Perspectives from World Englishes](https://md.research.elamur.ai/free/en/papers/ugwuanyi-2026-generative-ai-linguistic-diversity-academic/)
- [next → IdentifyMe: A Challenging Long-Context Mention Resolution Benchmark for LLMs](https://md.research.elamur.ai/free/en/papers/manikantan-2025-identifyme-challenging-long-context-mention/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/)

## References

21 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. We addressed the QIAS 2025 Shared Task using large language models with task-specific prompting strategies. For Subtask 1, live Google Search retrieval achieved 0.895 accuracy, while for Subtask 2, few-shot prompting reached 0.9259 accuracy. The main limitations include the dependence on the quality of the retrieval, the doctrinal differences, and the dependence on the closed-source Claude Opus 4 API. Future work will fine-tune Arabic-specific models and employ domain-restricted RAG over curated texts to mitigate coverage gaps and ambiguity. [find](https://scholar.google.com/scholar?q=We+addressed+the+QIAS+2025+Shared+Task+using+large+language+models+with+task%2Dspecific+prompting+strategies%2E+For+Subtask+1%2C+live+Google+Search+retrieval+achieved+0%2E895+accuracy%2C+while+for+Subtask+2%2C+fe)
2. Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur A Chowdhury, Fahim Dalvi, and 1 others. 2025. [Fanar: An arabic-centric multimodal generative ai platform](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/). CoRR.
3. Alaa N. Akkila and Samy S. Abu Naser. 2016. Proposed expert system for calculating inheritance in islam. [find](https://scholar.google.com/scholar?q=Alaa+N%2E+Akkila+and+Samy+S%2E+Abu+Naser%2E+2016%2E+Proposed+expert+system+for+calculating+inheritance+in+islam%2E)
4. Ahmet Yusuf Alan, Enis Karaarslan, and Ömer Aydin. 2024. A rag-based question answering system proposal for understanding islam: Mufassirqas llm. arXiv preprint arXiv:2401.15378. [↗](https://arxiv.org/abs/2401.15378)
5. Sarah Alnefaie, Eric Atwell, and Mohammad Ammar Alsalka. 2023. Is gpt-4 a good islamic expert for answering quran questions? In Proceedings of the 35th Conference on Computational Linguistics and Speech Processing (ROCLING 2023), pages 124–133. [find](https://scholar.google.com/scholar?q=Sarah+Alnefaie%2C+Eric+Atwell%2C+and+Mohammad+Ammar+Alsalka%2E+2023%2E+Is+gpt%2D4+a+good+islamic+expert+for+answering+quran+questions%3F+In+Proceedings+of+the+35th+Conference+on+Computational+Linguistics+and+Spee)
6. M Saiful Bari, Yazeed Alnumay, Norah A Alzahrani, Nouf M Alotaibi, Hisham A Alyahya, Sultan Al-Rashed, Faisal A Mirza, Shaykhah Z Alsubaie, Hassan A Alahmed, Ghadah Alabduljabbar, and 1 others. 2024. [Allam: Large language models for arabic and english](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/). arXiv preprint arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390)
7. Abdessalam Bouchekif, Samer Rashwani, Emad Mohamed, Mutaz Al-Khatib, Heba Sbahi, Shahd Gaben, Wajdi Zaghouani, Aiman Erbad, and Mohammed Ghaly. 2025a. [Qias 2025: Overview of the shared task on islamic inheritance reasoning and knowledge assessment](https://md.research.elamur.ai/free/en/papers/bouchekif-2025-qias-2025-overview-shared-task/). In Proceedings of The Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
8. Abdessalam Bouchekif, Samer Rashwani, Heba Sbahi, Shahd Gaben, Mutaz Al-Khatib, and Mohammed Ghaly. 2025b. [Assessing large language models on islamic legal reasoning: Evidence from inheritance law evaluation](https://md.research.elamur.ai/free/en/papers/bouchekif-2025-assessing-large-language-models-islamic/). In Proceedings of The Third Arabic Natural Language Processing Conference (ArabicNLP 2025), Suzhou, China. Association for Computational Linguistics.
9. Khalil Hennara, Sara Chrouf, Mohamed Motaism Hamed, Zeina Aldallal, Omar Hadid, and Safwan AlModhayan. 2025. [Kuwain 1.5 b: An arabic slm via language injection](https://md.research.elamur.ai/free/en/papers/hennara-2025-kuwain-1-5b-arabic-slm/). arXiv preprint arXiv:2504.15120. [↗](https://arxiv.org/abs/2504.15120)
10. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Abdulmohsen Alharthi, Bang An, Juncai He, and 1 others. 2023. [Acegpt, localizing large language models in arabic](https://md.research.elamur.ai/free/en/papers/huang-2024-acegpt-localizing-large-language-models/). arXiv preprint arXiv:2309.12053. [↗](https://arxiv.org/abs/2309.12053)
11. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, and 1 others. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in neural information processing systems, 33:9459–9474.
12. Rana Malhas, Watheq Mansour, and Tamer Elsayed. 2022. [find](https://scholar.google.com/scholar?q=Rana+Malhas%2C+Watheq+Mansour%2C+and+Tamer+Elsayed%2E+2022%2E)
13. Qur’an QA 2022: [Overview of the first shared task on question answering over the holy qur’an](https://md.research.elamur.ai/free/en/papers/malhas-2022-qur-qa-2022-overview-first/). In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur’an QA and Fine-Grained Hate Speech Detection, pages 79–87, Marseille, France. European Language Resources Association.
14. Rana Malhas, Watheq Mansour, and Tamer Elsayed. 2023. [find](https://scholar.google.com/scholar?q=Rana+Malhas%2C+Watheq+Mansour%2C+and+Tamer+Elsayed%2E+2023%2E)
15. Qur’an QA 2023 [shared task: Overview of passage retrieval and reading comprehension tasks over the holy qur’an](https://md.research.elamur.ai/free/en/papers/malhas-2023-qur-qa-2023-shared-task/). In Proceedings of ArabicNLP 2023, pages 690–701, Singapore (Hybrid). Association for Computational Linguistics.
16. Marryam Yahya Mohammed, Sama Ayman Ali, Salma Khaled Ali, Ayad Abdul Majeed, and Ensaf Hussein Mohamed. 2025. Aftina: enhancing stability and preventing hallucination in ai-based islamic fatwa generation using llms and rag. Neural Computing and Applications, pages 1–26. [↗](https://doi.org/10.1007/s00521-025-11229-y)
17. Faiza Qamar, Seemab Latif, and Rabia Latif. 2024. A benchmark dataset with larger context for non-factoid question answering over islamic text. arXiv preprint arXiv:2409.09844. [↗](https://arxiv.org/abs/2409.09844)
18. Mohammad Amaan Sayeed, Mohammed Talha Alam, Raza Imam, Shahab Saquib Sohail, and Amir Hussain. 2025. From rag to agentic: Validating islamic-medicine responses with llm agents. arXiv preprint arXiv:2506.15911. [↗](https://arxiv.org/abs/2506.15911)
19. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, and 1 others. 2023. [Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models](https://md.research.elamur.ai/free/en/papers/sengupta-2023-jais-jais-chat-arabic-centric/). arXiv preprint arXiv:2308.16149. [↗](https://arxiv.org/abs/2308.16149)
20. Sadia Tabassum, AHM Hoque, Sharaban Twahura, and Mohammad Osiur Rahman. 2019. Developing an islamic farayez system applying software engineering. Jurnal Kejuruteraan, 31(1):25–38. [↗](https://doi.org/10.17576/jkukm-2019-31(1)-04)
21. Samia Zouaoui and Khaled Rezeg. 2021. Islamic inheritance calculation system based on arabic ontology (arafamonto). Journal of King Saud University-Computer and Information Sciences, 33(1):68–76. [find](https://scholar.google.com/scholar?q=Samia+Zouaoui+and+Khaled+Rezeg%2E+2021%2E+Islamic+inheritance+calculation+system+based+on+arabic+ontology+%28arafamonto%29%2E+Journal+of+King+Saud+University%2DComputer+and+Information+Sciences%2C+33%281%29%3A68%E2%80%9376%2E)

*Updated 1 November 2026*