---
title: "PuxAI at QIAS 2025 — paper by Phuc and Văn (2025)"
description: "The paper describes a system for the QIAS 2025 shared task, which tests LLMs on Islamic sciences… 14 references, cited by 4 papers here."
authors: ["Nguyen Xuan Phuc", "Thìn \\DJặng Văn"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks"
source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.125/"
doi: "10.18653/v1/2025.arabicnlp-sharedtasks.125"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/phuc-2025-puxai-qias-2025-multi-agent/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/phuc-2025-puxai-qias-2025-multi-agent/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/12/) / PuxAI at QIAS 2025: Multi-Agent Retrieval-Augmented Generation for Islamic Inheritance and Knowledge Reasoning

# PuxAI at QIAS 2025: Multi-Agent Retrieval-Augmented Generation for Islamic Inheritance and Knowledge Reasoning

**Authors:** Nguyen Xuan Phuc and [Thìn \\DJặng Văn](https://md.research.elamur.ai/free/en/authors/thin-djang-van/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.125/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.125.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.125](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.125) · [OpenAlex](https://openalex.org/W4416037645)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/phuc-2025-puxai-qias-2025-multi-agent/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes a system for the QIAS 2025 shared task, which tests LLMs on Islamic sciences through multiple-choice questions. Subtask 1 covers inheritance law (ʿilm al-mawārīth), which requires identifying eligible heirs, applying fixed and residuary shares, and doing exact fractional arithmetic. Subtask 2 covers general knowledge across seven classical disciplines. The authors use a separate multi-step pipeline for each subtask, which they call Retrieval-Augmented Reasoning.

For inheritance, a "Virtual Inheritance Expert" first converts the case into JSON. It then uses that structure to retrieve related fatwas and reasons over both with few-shot chain-of-thought prompts. For knowledge questions, the retrieved passages are divided between a Proponent agent and a Critic agent, and a "head scholar" model makes the final decision. Three LLMs run each pipeline, and a majority vote picks the answer.

The system placed 2nd in Subtask 1 and 1st in Subtask 2. The authors argue that matching the pipeline design to the type of reasoning beats relying on a single monolithic model.

## Key points
- Subtask 1 data: about 20,000 train, 1,000 validation and 1,000 test MCQs, each with six options. The items were built from IslamWeb fatwas with Gemini 2.5 and checked by an expert. A further 3,165 fatwas serve as the retrieval base.
- Subtask 2 data: 1,400 four-option MCQs (700 validation, 700 test), drawn from 25 classical reference works and validated by five experts.
- Retrieval uses FAISS with BAAI/bge-m3 embeddings and top-k = 10.
- The ensemble combines Gemini-2.0-Flash, Fanar (Islamic-RAG) and Mistral (Saba-24B), with temperature 0.1 and majority voting. Gemini decides when all three models disagree.
- Test accuracy:
  - Subtask 1: 0.957, 2nd place (the winner, Gumball, scored 0.972).
  - Subtask 2: 0.9369, 1st place (Athar was next at 0.9272).

## Interesting ideas
- Turning the free-text case into a structured record (heirs, counts, relations) before retrieval makes the search query sharper and gives the reasoning step a canonical input.
- Dealing retrieved evidence by rank, odd ranks to one agent and even ranks to the other, gives both sides equally strong but different material, while the judge still sees everything.
- A majority vote whose tie-breaker is the member with the best measured validation accuracy is a simple, data-justified way to resolve a full split.
- The error analysis separates failures where the needed evidence was never retrieved from failures where it was present but misread, and separates wrong rule application from right numbers with wrong legal reasons.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (world): Retrieval of compiled knowledge (verb classes, measures, ontology facts) could be keyed on the extracted world of the text rather than on raw words, so lookups follow entities and holdings instead of surface strings.
- **I2** (learn): When an LLM teacher tier labels silver data, two teachers could each see different halves of the context, and a final arbiter would weigh their disagreements, exposing labels that depend on a single passage.
- **I3** (learn): Silver trees agreed by witness parsers could fall back to the parser with the best per-construction accuracy when no majority exists, instead of discarding the sentence.
- **I4** (answer): Test graders for school courses and word problems could label each miss as a missing-knowledge, misreading or wrong-rule error, which tells us whether to fix the world, the frames, or the solver.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Political Science and International Relations](https://md.research.elamur.ai/free/en/shelves/social-sciences/political-science-and-international-relations/) · 182 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Nguyen Xuan Phuc and Thìn \\DJặng Văn. 2025. PuxAI at QIAS 2025: Multi-Agent Retrieval-Augmented Generation for Islamic Inheritance and Knowledge Reasoning. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{phuc2025puxai,
  title = {{PuxAI at QIAS 2025: Multi-Agent Retrieval-Augmented Generation for Islamic Inheritance and Knowledge Reasoning}},
  author = {Nguyen Xuan Phuc and Thìn DJặng Văn},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.125},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.125}
}
```

- [← previous Predicting the Target Word of Game-playing Conversations using a Low-Rank Dialect Adapter for Decoder Models](https://md.research.elamur.ai/free/en/papers/srirag-2025-predicting-target-word-game-playing/)
- [next → Read Your Own Mind: Reasoning Helps Surface Self-Confidence Signals in LLMs](https://md.research.elamur.ai/free/en/papers/podolak-2025-read-own-mind-reasoning-helps/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/12/)

## References

14 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Hamdy Mubarak, Shammur Absar Chowdhury, Maram Hasanain, Basel Mousi, Sabri Boughorbel, Yassine El Kheir, Daniel Izham, Fahim Dalvi, Majd Hawasly, Nizi Nazar, Yousseif Elshahawy, Ahmed Ali, Nadir Durrani, Natasa Milic-Frayling, and Firoj Alam. 2024. [Larabench: Benchmarking arabic ai with large language models](https://md.research.elamur.ai/free/en/papers/abdelali-2024-larabench-benchmarking-arabic-ai-large/). Preprint, arXiv:2305.14982. [↗](https://arxiv.org/abs/2305.14982)
2. Alaa N. Akkila and Samy S. Abu Naser. 2016. Proposed expert system for calculating inheritance in islam. World Wide Journal of Multidisciplinary Research and Development, 2(9):38–48. [find](https://scholar.google.com/scholar?q=Alaa+N%2E+Akkila+and+Samy+S%2E+Abu+Naser%2E+2016%2E+Proposed+expert+system+for+calculating+inheritance+in+islam%2E+World+Wide+Journal+of+Multidisciplinary+Research+and+Development%2C+2%289%29%3A38%E2%80%9348%2E)
3. Abdessalam Bouchekif, Samer Rashwani, Mohammed Ghaly, Mutaz Al-Khatib, Emad Mohamed, Wajdi Zaghouani, Heba Sbahi, Shahd Gaben, and Aiman Erbad. 2025a. [Qias 2025: Overview of the shared task on islamic inheritance reasoning and knowledge assessment](https://md.research.elamur.ai/free/en/papers/bouchekif-2025-qias-2025-overview-shared-task/). In Proceedings of The Third Arabic Natural Language Processing Conference, ArabicNLP 2025, Suzhou, China, November 5–9, 2025. Association for Computational Linguistics.
4. Abdessalam Bouchekif, Samer Rashwani, Heba Sbahi, Shahd Gaben, Mutaz Al-Khatib, and Mohammed Ghaly. 2025b. [Assessing large language models on islamic legal reasoning: Evidence from inheritance law evaluation](https://md.research.elamur.ai/free/en/papers/bouchekif-2025-assessing-large-language-models-islamic/). In Proceedings of The Third Arabic Natural Language Processing Conference (Arabic-NLP 2025), Suzhou, China. Association for Computational Linguistics.
5. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Preprint, arXiv:2005.14165. [↗](https://arxiv.org/abs/2005.14165)
6. Devvrit, Minhao Cheng, Cho-Jui Hsieh, and Inderjit Dhillon. 2020. Voting based ensemble improves robustness of defensive models. Preprint, arXiv:2011.14031. [↗](https://arxiv.org/abs/2011.14031)
7. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Preprint, arXiv:2005.11401. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, and 13 others. 2023. Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. Preprint, arXiv:2308.16149. [↗](https://arxiv.org/abs/2005.11401)
8. Connor Shorten, Charles Pierse, Thomas Benjamin Smith, Erika Cardenas, Akanksha Sharma, John Trengrove, and Bob van Luijt. 2024. Structuredrag: Json response formatting with large language models. Preprint, arXiv:2408.11061. [↗](https://arxiv.org/abs/2408.11061)
9. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, Mike Schaekermann, Amy Wang, Mohamed Amin, Sami Lachgar, Philip Mansfield, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Aguera y Arcas, and 12 others. 2023. [Towards expert-level medical question answering with large language models](https://md.research.elamur.ai/free/en/papers/singhal-2023-towards-expert-level-medical-question/). Preprint, arXiv:2305.09617. [↗](https://arxiv.org/abs/2305.09617)
10. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, and 23 others. 2025. [Fanar: An arabic-centric multimodal generative ai platform](https://md.research.elamur.ai/free/en/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/). Preprint, arXiv:2501.13944. [↗](https://arxiv.org/abs/2501.13944)
11. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. MuSiQue: Multi-hop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554. [find](https://scholar.google.com/scholar?q=Harsh+Trivedi%2C+Niranjan+Balasubramanian%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2022%2E+MuSiQue%3A+Multi%2Dhop+questions+via+single%2Dhop+question+composition%2E+Transactions+of+the+Association+for+Computational+Li)
12. Xiaohan Xu, Chongyang Tao, Tao Shen, Can Xu, Hongbo Xu, Guodong Long, Jian guang Lou, and Shuai Ma. 2024. Re-reading improves reasoning in large language models. Preprint, arXiv:2309.06275. [↗](https://arxiv.org/abs/2309.06275)
13. Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, and 21 others. 2023. [Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models](https://md.research.elamur.ai/free/en/papers/guha-2023-legalbench-collaboratively-built-benchmark-measuring/). Preprint, arXiv:2308.11462. [↗](https://arxiv.org/abs/2308.11462)
14. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. [find](https://scholar.google.com/scholar?q=Patrick+Lewis%2C+Ethan+Perez%2C+Aleksandra+Piktus%2C+Fabio+Petroni%2C+Vladimir+Karpukhin%2C+Naman+Goyal%2C+Heinrich+K%C3%BCttler%2C+Mike+Lewis%2C+Wen+tau+Yih%2C+Tim+Rockt%C3%A4schel%2C+Sebastian+Riedel%2C+and+Douwe+Kiela%2E+2021%2E)

*Updated 31 October 2026*